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commands  do  not  have  the  infrastructure  or  funding  that  is  required  to  support  the  rapid 
expansion  of  multiple  phone  services  at  various  locations  throughout  the  world.  VoIP 
offers  a  rapidly  deployable  alternative.  A  subjective  study  was  designed  to  test  the  quality 
of  Voice  over  Internet  Protocol  (VoIP)  signals  in  a  controlled  and  isolated  multilevel 
secure  network  to  which  single  level  networks  were  attached.  The  experiment  provided 
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GLOSSARY 


ARP:  Address  Resolution  Protocol  is  a  core  protocol  in  the  Transmission  Control 
Protocol/Intemet  Protocol  (TCP/IP),  and  is  used  to  obtain  the  physical  address  of  a  host 
or  node.  1 

CONFIDENCE  INTERVAL:  The  confidence  interval  is  an  estimate  of  a  population 
parameter  that  consists  of  a  range  of  values  bound  by  statistics  referred  to  as  “upper”  and 
“lower”  confidence  limits,  within  which  the  value  of  the  parameter  is  expected  to  be 
located.2 

CONFIDENCE  LEVEL:  The  confidence  level  is  an  expression  that  represents  how  often 
the  true  percentage  of  the  population  would  choose  an  answer  which  lies  within  the 
confidence  interval’s  upper  and  lower  limits.3 

ENCODING:  The  process  of  transforming  information  from  one  format  into  another 
format.  4 

GATEWAY :  A  router  that  connects  two  networks  and  can  perform  protocol 
conversion.5 

IEC:  International  Electro-technical  Commission  is  an  international  standards  and 
conformity  assessment  body  for  all  fields  of  electro-technology. 


1  Tamara  Dean,  Network  +  Guide  to  Networks,  Thomson  Course  Technology,  2006. 

2  Creative  Research  Systems,  The  Survey  System  (URL);  http://www/surveysystem.com/sscalc.htm, 
[Accessed:  February  20,  2008]. 

3  Ibid. 

4  Charlie  Kaufman,  Radia  Perlman,  Mike  Speciner,  Network  Security  Private  Communication  in  a 
Public  World,  Prentice  Hall  PTR,  1995. 

5  Mani  Subramanian,,  Network  Management,  Principles  and  Practice,  Addison- Wesley,  2000. 
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IPv4:  Internet  Protocol  version  4  is  the  fourth  iteration  of  the  Internet  Protocol  and  is  the 
Current  standard  for  IP  addressing.  It  specifies  32-bit  addresses  composed  of  four  octets.6 

IPv6:  Internet  Protocol  version  6  (IPv6)  is  a  newer  standard  Internet  Protocol  addressing 
that  will  replace  the  IPv4.  IPv6  uses  a  newer,  more  efficient  header  in  its  packets  and 
allows  for  128-bit  source  and  destination  IP  addresses.  The  addition  of  longer  addresses 
will  allow  for  more  addresses.7 

ISO:  International  Organization  of  Standardization  is  an  international  standard-setting 
body  composed  of  representatives  from  various  national  standards  organizations.8 

ITU:  International  Telecommunications  Union  is  a  United  Nations  agency  that  regulates 
international  telecommunications  and  provides  developing  countries  with  technical 
expertise  and  equipment  to  advance  their  technological  bases.9 

ITU-T:  International  Telecommunication  Union  Telecommunications  Standardization 
Sector  is  a  permanent  subdivision  of  the  International  Telecommunications  Union  (ITU) 
and  is  responsible  for  providing  standards  by  studying,  and  operating  telecommunication 
networks.10 

MAC:  Media  Access  Control  address  is  a  12  character  string  that  uniquely  identifies  a 
network  node.11 


6  Tamara  Dean,  Network  +  Guide  to  Networks,  Thomson  Course  Technology,  2006. 

7  Ibid. 

8  Charlie  Kaufman,  Radia  Perlman,  Mike  Speciner,  Network  Security  Private  Communication  in  a 
Public  World,  Prentice  Hall  PTR,  1995. 

9  Tamara  Dean,  Network  +  Guide  to  Networks,  Thomson  Course  Technology,  2006. 

10  International  Telecommunications  Union  Standardization  Sector  (ITU-T),  General  Information, 
http://www.itu.int/ITU-T/info/index.html,  [Accessed:  February  14,  2008], 

1 1  Tamara  Dean,  Network  +  Guide  to  Networks,  Thomson  Course  Technology,  2006. 
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MLS:  Multilevel  Security:  A  class  of  systems  containing  information  at  different 
classification  levels.  Access  decisions  are  based  on  the  subject’s  security  clearances,  need 
to  know  and  formal  approval.12 

MYSEA:  The  Monterey  Security  Architecture  is  a  trusted,  distributed  environment 
enforcing  multilevel  security  policies. 

NIPRNet:  Unclassified  but  Sensitive  Internet  Protocol  Router  Network,  (formerly  called 
the  Non-Classified  Internet  Protocol  Router  Network).  NIPRNet  is  used  to  exchange 
unclassified  but  sensitive  information  between  "internal"  users  and  also  provides  user 
access  to  the  Internet. 

PSTN:  The  Public  Switched  Telephone  Network  is  the  network  of  the  world's  public 
circuit-switched  telephone  networks. 

QoS:  Quality  of  Service  is  the  ability  to  guarantee  a  certain  level  of  performance  for  a 
data  flow.13 

SIP:  Session  Initiation  Protocol  is  an  application-layer  control  (signaling)  protocol  for 
creating,  modifying,  and  terminating  sessions  with  one  or  more  participants.  These 
sessions  include  Internet  telephone  calls  (VoIP),  multimedia  distribution,  and  multimedia 
conferences.14 

SIPRNet:  The  SIPRNet  (Secret  [formerly  Secure]  Internet  Protocol  Router  Network)  is  a 
system  of  interconnected  computer  networks  used  by  the  U.S.  Department  of  Defense 
and  the  U.S.  Department  of  State  to  transmit  classified  information  (up  to  and  including 


12  Shon  Harris,  CISSP  All-in-One  Guide  Third  Edition,  McGraw-Hill,  2005. 

13  Ibid. 

14  J.  Rosenbury,  The  Internet  Engineering  Task  Force  (IETF),  Request  For  Comments  (RFC)  3261, 
The  Internet  Society,  2002,  http://www.ieft.org/rfc/rfc3261.txt,  [Accessed:  February  14,  2008], 

xvii 


information  classified  SECRET)  in  an  identification  and  authentication  environment  that 
is  isolated  from  the  NIPRNet  through  a  combination  of  logical  and  physical  measures. 

SSO:  Single  Sign-On  (SSO)  is  a  method  of  access  control  that  enables  a  user  to 
authenticate  once  and  gain  access  to  the  resources  of  multiple  software  systems.15 

TCP/IP:  Transmission  Control  Protocol/Intemet  Protocol  is  a  suite  of  networking 
protocols  that  provides  the  foundation  for  data  exchange  across  the  Internet.16 

TPE:  Trusted  Path  Extension  is  a  device  responsible  for  providing  a  secure  interface  for 
user  interaction  with  selected  MYSEA  server  security  functions.17 

VoIP:  Voice  over  Internet  Protocol  is  a  protocol  for  providing  telephone  service  over  a 
packet-switched  network  running  the  TCP/IP  protocol  suite.18 

WTSA:  World  Telecommunication  Standardization  Assembly  defines  general  policy  and 
adopts  working  methods  and  procedures  for  the  ITU-T. 

XML:  The  Extensible  Markup  Language  (XML)  is  a  general-purpose  markup  language. 
It  is  used  on  the  World  Wide  Web  in  the  context  of  the  Hyper-Text  Transfer  Protocol.  19 


15  Shon  Harris,  CISSP  All-in-One  Guide  Third  Edition,  McGraw-Hill,  2005,  149-151. 

16  Tamara  Dean,  Network  +  Guide  to  Networks,  Thomson  Course  Technology,  2006. 

17  Thuy  D.  Nguyen,  Timothy  E.  Levin,  Cynthia  E.  Irvine,  MYSEA  Testbed,  2004. 

18  Tamara  Dean,  Network  +  Guide  to  Networks,  Thomson  Course  Technology,  2006. 

19  E.  Whitehead,  The  Internet  Engineering  Task  Force  (IETF),  Request  For  Comments  (RFC)  2376, 
The  Internet  Society,  1998,  <http://www.ieft.org/rfc/rfc2376.txt>,  [Accessed:  February  14,  2008], 
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I. 


INTRODUCTION 


Voice  over  Internet  Protocol  (VoIP)  is  a  growing  technology  that  has  been 
adopted  and  used  by  various  components  of  the  commercial  sector.  As  with  most  of 
today’s  advanced  technology,  the  military  is  adopting  this  new  digital  communication 
concept,  with  some  of  the  military  services  planning  to  implement  VoIP  in  ground  and 
sea  deployments  to  provide  integrated  communications  at  all  levels  of  the  command 
structure20. 

The  primary  focus  of  utilizing  VoIP  technology  is  to  provide  voice 
communication  down  to  the  lowest  level  within  the  command.  This  technology  will 
permit  data  exchange  between  separate  commands  without  the  need  to  deploy  legacy 
Public  Switched  Telephone  Networks  (PSTN)  equipment  or  infrastructures  regardless  of 
the  location.  When  a  command  is  deployed  and  interconnected  to  the  main  information 
grid,  members  of  the  command  are  able  to  communicate  with  one  another,  as  well  as  with 
senior  authorities  as  needed. 21 

As  the  term  Voice  over  IP  implies,  it  is  a  PSTN  replacement  that  enables 
connectivity  to  go  beyond  just  speech  communication.  VoIP  can  be  used  for 
automatically  sending  additional  information  in  the  form  of  chat,  video  streaming, 
application  files,  and  other  web-based  application  sharing  features. 

Even  though  the  immediate  deployment  of  VoIP  in  battle  conditions  is  more 
spectacular  and  draws  attention,  VoIP’s  beneficial  contribution  to  permanent  installations 
such  as  naval  bases  or  naval  air  stations  must  not  be  downplayed.  It  is  easy  to  see  why 
VoIP  should  be  considered  for  wide  deployment,  especially  considering  the  cost  savings 


20  Maryann  Lawlor,  Jeff  Hawk,  Henry  S.  Kenyon,  Panelists  Explore  Network  Centricity’s  Many 
Facets,  SIGNAL  AFCEAS;  International  Journal,  August  2005, 

http://www.afcea.org/signal/articles/templates/SIGNAL_Article_Temlate.asp7articleicE101 18&zoneid=8, 
[Accessed:  March  21,  2007], 

21  CMDR  Jeffrey  W.  Eggers,  Networks  and  Knowing,  Armed  Forces  Journal,  February  2008. 
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compared  to  traditional  public  switched  telephone  networks  (PSTN).  The  installation  cost 
is  also  decreased  since  the  existing  network  infrastructure  is  used  and  manpower  skills 
needed  for  administration  are  reduced22 

The  real  evolution  in  military  communications  will  come  from  the  combined  use 
of  VoIP  within  multilevel  secure  and  non-multilevel  secure  networks.  A  deployed 
command  will  not  need  to  carry  copper/fiber  cable,  to  enable  communications.  All 
commands  can  be  interconnected  without  physical  cabling  linking  them,  and  all  service 
personnel  can  communicate  seamlessly.  First,  the  man-hours  and  skills  needed  to  deploy 
and  effectively  administer  the  network  are  minimal,  which  is  an  important  factor  when 
the  available  manpower  is  limited  and  time  is  crucial.  Second,  it  allows  for  further 
network  expansion  when  additional  troops  arrive  and  the  need  for  communication 
between  networks  is  urgent23. 

A.  THESIS  OBJECTIVE 

This  thesis  explores  the  quality  of  Voice  over  Internet  Protocol  (VoIP),  when 
configured  within  a  multilevel  secure  network.  Subjective  testing  has  been  used  to 
compare  the  results  obtained  in  four  network  configurations,  two  multilevel  secure  and 
two  non-multilevel  secure  networks. 

The  International  Telecommunications  Standardization  Sector  (ITU-T)  P.862.3,24 
was  used  as  the  standard  in  the  subjective  testing  of  voice  quality  within  VoIP  signals  in  a 
controlled  and  isolated  multilevel  network  to  which  networks. 

The  Monterey  Secure  Architecture,  MYSEA,  was  the  test  bed  for  the  two 
multilevel  secure  networks.  The  two  non-multilevel  secure  networks  had  similar 
configurations  and  all  networks  had  VoIP  software,  Skype,  installed. 

Human  test  subjects  were  solicited  from  the  staff  and  students  of  the  Naval 
Postgraduate  School  to  participate  in  the  experiments.  Each  test  subject  was  briefed  and 


22  Karen  E.  Thuermer,  Gearing  up  for  Networx,  Military  Information  Technology,  Vol  1 1,  Issue  8, 
September  25,  2007. 

23  B.  Sklar,  Digital  Communications:  Fundamental  and  Applications,  Prentice  Hall,  Upper  Saddle 
River,  2001. 

24  ITU-T  Recommendation  P.862  (2001)  -  Amendment  2. 
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instructed  to  listen  to  sound  files  on  each  network  configuration  and  complete  a  separate 
ten-item  questionnaire  regarding  each  sound  file.  Each  questionnaire  was  graded  and 
tabulated  using  a  Mean  Opinion  Score  (MOS),  which  were  used  for  single  and  multi 
attribute  analysis. 

The  hypothesis  examined  in  this  thesis  is,  “The  quality  of  voice  transmission  over 
Internet  Protocol  networks  in  a  multilevel  secure  network  is  comparable  to  a  non¬ 
multilevel  secure  network.” 
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II.  BACKGROUND 


A.  OVERVIEW 

This  chapter  presents  background  information  pertaining  to  this  thesis  study.  The 
focus  is  the  qualitative  assessment  of  Voice  over  Internet  Protocol  (VoIP)  within  the 
multilevel  secure  network  of  the  MYSEA  (Monterey  Security  Architecture).  The 
assessment  method  used  is  based  on  the  testing  guidelines  and  standardization  of  the 
International  Telecommunications  Union  Standardization  Sector  (ITU-T). 

Although  VoIP  is  an  alternative  to  public  switched  telephone  network  (PSTN) 
services  with  respect  to  cost  and  flexibility,  the  quality  of  its  output  can  determine 
whether,  and  under  what  conditions,  VoIP  will  be  acceptable  to  its  end  users  when 
operated  within  a  multilevel  secure  environment.  This  thesis  will  answer  the  following 
question:  What  is  the  quality  of  voice  transmission  over  IP  networks  in  a  multilevel 
secure  environment  when  compared  to  the  same  service  in  a  non-multilevel  secure 
environment? 

B.  INTERNATIONAL  TELECOMMUNICATIONS  UNION 

The  United  Nations  refers  to  the  International  Telecommunication  Union  (ITU) 
for  specialized  information  and  standards  in  the  field  of  telecommunications.  The  ITU 
Telecommunication  Standardization  Sector  (ITU-T)  is  a  permanent  subdivision  of  ITU.  It 
is  the  ITU-T  division’s  responsibility  to  study  technical  configurations,  operating 
procedures  and  resolve  questions  along  with  issuing  recommendations  on  each  proposed 
issue  and  to  provide  a  technical  viewpoint  on  standardizing  telecommunications  on  a 
worldwide  basis.  Another  related  organization,  The  World  Telecommunication 
Standardization  Assembly  (WTSA),  meets  every  four  years,  and  establishes  all  of  the 
topics  for  study  by  the  ITU-T  study  groups.  The  ITU-T  will  then,  produce 
recommendations  on  each  of  these  topics.  In  some  areas  of  information  technology  which 
fall  within  ITU-T's  area  of  responsibility,  standards  are  prepared  by  a  collaborative 
foundation  including  the  International  Organization  of  Standardization  (ISO)  and  The 
International  Electro-technical  Commission  (IEC). 
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The  International  Telecommunications  Union  Standardization  Sector  (ITU-T)  is 
the  source  of  the  standards  for  all  test  configurations  and  implementations  used  for  this 
thesis  study.  The  ITU’s  standards  date  back  to  1865,  the  date  of  formation  of  the 
International  Telegraph  Union.  The  ITU-T  has  been  an  intergovernmental  public-private 
partnership  organization  since  its  inception.  It  became  a  specialized  agency  under  the 
United  Nations  in  1947.  It  has  a  membership  of  191  countries  and  over  700  public  and 
private  sector  companies,  as  well  as  international  and  regional  telecommunication 
entities.25  It  was  renamed  in  1993  to  its  current  title:  International  Telecommunications 
Union  Standardization  Sector  (ITU-T).  Its  mission  is  to  ensure  efficient  and  timely 
creation  of  standards  covering  all  fields  of  telecommunications  on  a  worldwide  basis,  as 
well  as  defining  bookkeeping  principles  for  international  telecommunications  services. 
ITU-T’s  use  increased  in  the  personal  computer  industry  in  the  early  1980’s,  often 
endorsing  ‘We  facto  standards”  as  the  technology  was  under  development.  The  ITU-T 
developed  a  streamlined  process  that  minimized  the  time  between  initial  proposals  and 
the  final  approval.  Sometimes  the  time  frame  is  as  short  as  three  months.  The  ITU-T 
technical  work  is  managed  by  multiple  study  groups.  Those  involved  in  the  study  groups 
are  experts  in  telecommunications  from  all  over  the  world. 

C.  MYSEA 

The  Monterey  Security  Enhanced  Architecture  (MYSEA)  was  created  at  the 
Naval  Postgraduate  School  to  provide  a  trusted  distributed  operating  environment  that 
enforces  multi-domain  security  policies,  and  supports  commercial  off  the  shelf  (COTS) 
applications.  MYSEA  provides  an  isolated  networking  environment  for  the  enforcement 
of  stringent  mandatory  security  policies.  The  architecture  consists  of  a  network  that 
operates  with  high  assurance  MYSEA  servers,  low  assurance  MYSEA  clients,  legacy 
single  level  networks,  and  dynamic  security  services.  MYSEA  supports  a  secure  trusted 
path  for  communications  between  the  user  and  the  trusted  operating  system. 

The  trusted  path  of  the  MYSEA  system  is  enabled  by  a  Trusted  Path  Extension 
(TPE)  device  which  creates  a  protected  channel  between  itself  and  the  MYSEA  server. 

25  International  Telecommunication  Union.  (2005).  Telecommunication  indicators  Handbook. 

Geneva:  ITU.  <http://www.itu.int/ITU-D/ict/material/handbook.pdf>,  [Accessed:  January  16,  2007]. 
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The  assurance  properties  of  the  TPE  device  ensure  that  the  trusted  path  cannot  be 
undermined  by  malware  on  any  of  the  client  workstations.  Architecturally,  the  Trusted 
Path  Extension  provides  the  client’s  only  access  to  the  MYSEA  network  for  each  logged 
on  user  and  at  the  user’s  sensitivity  level. 

The  MYSEA  test  bed  continues  to  evolve  as  a  growing  technology  that  includes: 

a.  Assured  authentication  and  trusted  paths  to  MLS  services,  (e.g.,  Email,  Web- 
based  Distributed  Authoring  and  Versioning  (DAV),  etc.). 

b.  Application  of  high  assurance  system-security  technology  to  the  integration  of 
commercial  and  legacy  components. 

c.  Centralized  security  management. 

d.  Integration  of  high  assurance  MLS  with  existing  sensitive  networks  (e.g., 
SIPRNET,  etc.). 

e.  High  assurance  trusted  communication  channel  techniques  for  managing 
access  to  various  classified  networks. 

f.  Use  of  open  architectures  and  standards. 

g.  Experimentation  on  the  use  of  an  extensible  markup  language  (XML)  tags  as 
security  markings. 

h.  Design  of  secure  single  sign-on  (SSO)  across  multiple  MLS  servers. 

i.  High  assurance  module  authentication  and  verification. 

j.  Secure  collaborative  information  sharing. 

In  order  to  maintain  a  strict  and  stable  life  cycle  process  within  the  MYSEA  test 
bed,  only  authorized  personnel  are  granted  access  to  the  multilevel  networks  and  only 
secure  hardware  components  and  software  releases  are  installed.26 

D.  VOIP 

Voice  over  Internet  Protocol  (VoIP)  was  first  introduced  in  1995  by  Vocal  Tec 
Inc.,  a  telecommunication  company  that  had  launched  a  multimedia  PC-based  product 
which  allowed  users  to  speak  through  a  PC  microphone  and  listen  to  the  conversation 


26  T.  D.  Nguyen,  T.  E.  Levin  ,  and  C.  E.  Irvine,  "MYSEA  Test  bed",  Proceedings  from  the  6th  IEEE 
Systems,  Man  and  Cybernetics  Information  Assurance  Workshop,  West  Point,  NY,  June  2005, 438-439. 
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from  the  PC’s  speakers.  This  product  was  referred  as  the  Internet  Phone.  VoIP  uses 
Internet  Protocols  as  the  data  transmission  vehicle.  A  VoIP  system  digitizes  voice  using 
an  audio  codec,  divides  the  digitized  voice  into  packets,  and  sends  the  packets  over  an  IP 
network  to  the  intended  destination.  All  packets  are  routed  through  the  network  so  they 
will  travel  the  same  path.  Unlike  a  public  switched  telephone  network  (PSTN)  or  legacy 
phone  call,  no  dedicated  circuit  is  ever  created  for  a  VoIP  call.  The  exact  process  required 
to  set  up  a  VoIP  call  is  dependent  on  the  VoIP  protocol.  Two  types  of  protocols  are 
necessary  to  complete  a  VoIP  call: 

1.  Signaling,  this  has  the  function  of  establishing  a  session  between  the  callers. 

2.  Media  transport,  which  specifies  the  rules  and  formats  of  the  actual  voice 
packets. 

For  PSTN  systems,  a  phone  number  is  used  to  locate  a  phone  via  the  switching 
network,  whereas,  a  phone  number  in  VoIP  can  be  a  regular  PSTN  phone  number,  an  IP 
address,  or  an  alias  destination  assignment.  The  “phone  number”  ultimately  is  translated 
to  a  32-bit  or  128-bit  IP  address,  which  is  dependent  on  whether  IPv4  or  IPv6  is  used. 
Every  VoIP  signaling  protocol  must  be  capable  of  providing  address  resolution  in  order 
for  each  call  to  reach  its  destination.  There  are  four  general  VoIP  communication  modes 
that  are  dependent  upon  this  address  resolution  protocol  (ARP): 

1 .  Phone -to-Phone, 

2.  Phone-to-PC 

3.  PC-to-Phone 

4.  PC-to-PC 

Voice  transmission  is  carried  by  both  PSTN  and  IP  networks  under  the  first  three 
modes,  (Phone-to-Phone,  Phone-to-PC,  and  PC-to-Phone).  A  VoIP  service  provider  that 
interconnects  the  PSTN  and  VoIP  networks  is  needed  for  the  first  three  modes  when  a 
call  originates  from  a  PSTN  network  and  arrives  at  a  VoIP  network  or  vice  versa.  Voice 
travels  exclusively  across  the  IP  network  in  the  fourth  mode  (PC-to-PC). 

The  VoIP  technology  is  made  up  of  five  distinct  services: 

1 .  Signaling 

2.  Encoding 
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3.  Transport 

4.  Session  Initiation  Protocol 

5.  Gateway  Control 

A  signaling  VoIP  protocol  establishes  and  manages  a  connection  between  the 
endpoints  when  a  call  is  made;  this  service  requires  the  use  of  the  VoIP  signaling 
protocol.  Session  Initiation  Protocol  (SIP)  is  used  as  the  media  transport  protocol  and  is 
used  for  setting  up  and  breaking  down  voice  calls.  A  primary  objective  in  using  SIP  is  to 
provide  a  connection  and  call  setup  protocol  for  IP -based  communications  that  can 
support  a  superset  of  the  call  processing  functions  and  features  present  in  the  public 
switched  telephone  network  (PSTN).  These  features  permit  telephone-like  operations 
such  as:  dialing  a  number,  enabling  a  telephone  to  ring,  providing  ring  tones  or  a  busy 
signal.  Implementation  and  terminology  are  different  in  the  SIP  world  but  to  the  end-user, 
the  behavior  is  similar  to  that  of  PSTN. 

SIP  is  a  peer-to-peer  protocol,  where  it  requires  only  a  simple  core  network  with 
intelligence  signals  distributed  throughout  the  network.  Although  many  other  VoIP 
signaling  protocols  exist,  SIP  is  characterized  as  having  its  roots  in  the  IP  community 
rather  than  the  telecom  industry.  When  the  conversation  takes  place,  voice  has  to  be 
encoded  before  it  is  transmitted  over  the  IP  network.  The  encoded  voice  packets  will  then 
be  transported  via  the  IP  network  to  the  destination.  A  gateway  may  be  needed  to  convert 
voice  into  another  format  suitable  for  the  receiving  network.27 


27  Uyless  D.  Black,  Voice  over  IP,  2nd  Edition,  Upper  Saddle  River,  NJ,  Prentice  Hall,  2002,  330. 

9 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


10 


III.  EXPERIMENTAL  DESIGN 


This  chapter  describes  the  test  methodology  used  to  evaluate  the  quality  of  the 
audio  content  transmitted  using  VoIP  in  different  network  architectures.  These 
architectures  included  a  typical  infrastructure  and  the  MYSEA  infrastructure. 

Testing  was  conducted  on  a  dedicated  test  bed  in  the  basement  of  Glasgow  East 
and  was  based  on  a  subjective  approach  using  human  test  subjects.  Each  test  subject 
answered  a  ten  item  questionnaire  after  listening  to  a  sound  file.  This  process  was 
repeated  for  each  of  four  different  network  configurations.  All  test  data  were  collected 
from  each  test  subject  and  retained  for  analysis. 

An  initial  experiment  (Experiment  I)  revealed  a  potential  bias  in  one  of  the  four 
sound  files.  The  initial  testing  was  conducted  with  61  test  subjects  presented  with  of  four 
separate  network  configurations,  where  they  listened  to  four  separate  sound  files  (sound 
files  A,  B,  C,  and  D).  However,  each  sound  file  remained  at  the  same  test  station 
throughout  the  entire  testing  process.  A  review  of  the  results  indicated  that  Station  Two 
displayed  significantly  higher  scores  than  the  remaining  three  stations.  This  indicated  the 
need  for  an  additional  experiment  to  identify  any  potential  sound  file  bias. 

A  second  experiment  (Experiment  II)  was  administered.  It  consisted  of  one  basic 
network  configuration  at  each  of  two  separate  testing  stations.  Forty  new  test  subjects 
were  solicited  to  participate  in  this  experiment.  For  four  test  groups,  the  same  four  sound 
files  were  played  in  different  order,  (e.g.,  Trial  1:  ABCD;  Trial  2:  BCDA;  Trial  3: 
CDAB;  Trial  4:  DABC).  The  results  of  the  second  experiment  indicated  that  Sound  File 
B,  had  higher  scores  than  the  other  three  sound  files  regardless  of  the  order  in  which  they 
were  presented. 

The  results  of  Experiment  II  revealed  that  Station  Two  in  Experiment  I  had  a 
sound  file  of  higher  quality.  Sound  File  B  words  were  more  understandable  and  had  an 
overall  better  quality  than  the  other  three  files.  With  this  bias  revealed,  the  results  from 
Station  Two  were  not  used  and  Station  One  was  the  baseline  for  comparison  of  Stations 
Three  and  Four  in  Experiment  I. 
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A.  EXPERIMENT  I 

The  objective  of  this  experiment  was  to  measure  voice  quality  using  standard 
sound  files  downloaded  from  the  International  Telecommunications  Union 
Standardization  Sector  (ITU-T)  website  and  transmitted  over  four  separate  test 
configurations  using  Skype  software.28  Two  test  configurations  included  the  MYSEA 
infrastructure  for  the  multilevel  secure  network  and  two  test  configurations  did  not 
include  the  MYSEA  infrastructure.  The  main  objective  of  the  tests  described  in  the 
following  subsections  was  to  determine  if  VoIP  voice  quality  changes  between  MLS  and 
non-MLS  networks. 

1.  Subject  Sampling 

Speech  quality  was  determined  as  the  result  of  a  subjective  test,  where  a  pre¬ 
determined  number  of  test  subjects  listen  to  and  judge  the  test  material  over  separate 
networks  via  a  ten-item  questionnaire.  All  test  subjects  were  random  volunteers  from  the 
students  and  staff  associated  with  the  Naval  Postgraduate  School.  The  sample  size  of  61 
test  subjects  were  selected  from  a  total  population  of  1500  staff  and  students.29  For  a 
confidence  level  of  95%  this  sample  size  corresponds  to  a  12%  confidence  interval.30 

Subjects  were  solicited  through  on-campus  announcements  which  were  posted  at 
various  academic  buildings  throughout  campus,  email  solicitations  from  academic 
advisors/professors  and  classmates.  These  methods  were  the  basis  of  success  in  obtaining 
the  required  sample  size  for  both  experiments. 

2.  Test  Question  Selection 

Testing  methods  were  centered  on  the  objective  of  defining  the  subjective 
performance  assessment  of  the  quality  of  speech  of  VoIP  in  four  separate  network 
configurations.  The  method  was  a  “listening  test”,  where  messages  were  presented 


28  ITU-T  Recommendation  P.862  (2001)  -  Amendment  2. 

29  Fred  N.  Kerlinger,  Foundations  of  Behavioral  Research,  3rd  Edition,  (CBS  College  Publishing, 
1986),  117-119. 

30  Creative  Research  Systems,  The  Survey  System  (URL);  http://www/surveysystem.com/sscalc.htm, 
[Accessed:  February  20,  2008]. 
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aurally  to  all  test  subjects.  Each  test  subject  was  given  a  ten-item  questionnaire  to  answer 
at  the  conclusion  of  each  sound  file  played  at  its  associated  station.  Each  of  the  test 
subject’s  opinions  were  rated  on  a  cardinality  rating  scale  of  0  to  5,  with  5  being  of 
greater  quality.  Appendix  A  shows  how  the  subjective  opinions  were  mapped  to 
cardinality  scores.  The  cardinality  point  system  was  not  known  to  the  subject.  These 
results  were  measures  of  the  perceived  quality  in  four  categories  of  perception:  Listening- 
quality  effort,  Articulation,  Acceptance,  and  Overall  Impression.  The  listening  scores 
made  it  possible  to  compare  the  effectiveness  of  different  speech  files  over  each  of  the 
four  separate  network  configurations.  The  questionnaires  used  throughout  both 
experiments  (I  and  II)  were  the  same  and  are  included  in  Appendix  A.31 

3.  Sound  File  Selection 

ITU-T  Recommendation  P.862.3  was  used  to  provide  a  guide  for  estimating 
listening  speech  quality  by  using  reference  sound  files.  This  recommendation  provides 
the  necessary  information  for  obtaining  a  stable  and  reliable  reference  for  the  listener’s 
perception  of  speech  quality. 

It  was  recommended  that  the  speech  material  consist  of  simple,  short  and 
meaningful  sentences.  These  sentences  were  chosen  to  be  easy  to  understand  and  were 
constructed  into  sets  of  two  short  sentences  with  no  obvious  connection  of  meaning 
between  the  sentences  in  a  set.  Additionally,  in  accordance  with  P.830,  it  was 
recommended  that  a  minimum  of  two  female  and  two  male  sound  files  be  used  for 
testing.32 

4.  Controlling  Station 

The  experiment  controller  played  the  sample  files  for  all  test  subjects  prior  to 
testing  at  each  station  for  sound  and  accent  familiarity.  Additionally,  he  initiated  all 


31  ITU-T  Recommendation  P.85,  Telephone  Transmission  Quality  Subjective  Opinion  Tests,  June 
1994. 

32  ITU-T  Recommendation  P.862.3,  Telephone  Transmission  Quality,  Telephone  Installations,  Local 
Line  Networks,  November  2005. 
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telephone  connections  between  the  controlling  station  and  each  of  the  testing  stations  via 
Skype  VoIP  software.  The  experiment  controller  used  the  same  equipment  for 
Experiments,  I  and  II. 

a.  Equipment:  Dell  Desktop  CPU  (Dimension  8200);  four  individual  ITU 
pre-recorded  voice  sound  files,  (two  male,  two  female)  played  via  RealPlayer 
Enterprise  Player,  Build:  6.0.11.1526,  Copyright  (c)  2004;  (1)  4  ft,  3.5  mm 
stereo  audio  cable  (Male/Male). 

b.  Network:  Dell  Desktop  CPU  connected  to  the  Skype  server  via  the  Naval 
Postgraduate  School  Internet  (unclassified  Internet). 

c.  Setup:  A  standard  3.5mm  stereo  audio  cable  (Male/Male)  was  connected 
to  the  speaker  output  jack  (back  of  PC)  and  to  the  microphone  input  jack  (back 
of  PC)  to  create  a  loopback  circuitry  in  order  for  the  sound  file  to  be 
transmitted  to  the  Skype  Server  and  back  to  the  connected  client  for  each  of 
the  four  station  configurations. 

NOTE:  This  testing  was  targeted  for  single  connectivity,  therefore,  no  conference  calling 
features  were  tested  or  evaluated.  All  calls  were  made  to  one  specific  station  at  a  time  and 
only  one  file  was  transmitted  over  that  connection. 

5.  Station  One 

a.  Equipment:  Dell  Inspiron  Notebook  CPU  (5150);  Plantronics  PC  Headset. 

b.  Network:  Dell  Desktop  CPU  (Dimension  8200)  physically  connected  to  the 
NPS  Internet  using  CAT  5  cabling. 

c.  Setup:  ITU  pre-recorded  wave  file  played  and  transmitted  from  the 
experiment  controller’s  desktop  to  the  test  subject’s  headset  using  Skype 
software. 

d.  Configuration:  As  shown  in  Figure  1  there  was  no  intermediate  MYSEA 
Server  (direct  connection  to  the  Internet  and  the  Skype  Server).  In  this 
configuration,  Station  One  is  connected  directly  to  the  Internet,  via  NPS 
Internet,  using  Skype  software  installed  on  the  Dell  Inspiron  Notebook.  Prior 
to  testing,  the  experiment  controller  ensured  that  the  laptop  was  logged  on  and 

connected  to  the  Internet.  The  experiment  controller  opened  and  activated  the 
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Skype  account  for  “Non-MLS  One”  client.  After  logging  onto  the  Skype 
Server,  the  experiment  controller  initiated  the  call  from  the  Dell  desktop  via 
the  Skype  Server.  After  the  call  was  initiated  and  received  by  the  “Non-MLS 
One”  client,  connectivity  was  established.  Each  test  subject  listened  to  File 


Figure  1.  Station  One 


6.  Station  Two 

a.  Equipment:  Dell  Inspiron  Notebook  CPU  (5150);  Plantronics  PC  Headset. 

b.  Network:  Dell  Desktop  CPU  (Dimension  8200)  physically  to  the  NPS  Internet 
with  the  Dell  Inspiron  Notebook  CPU  (5150)  connected  to  a  Proxy  Server 
(Dell  Desktop  CPU-Dimension  8200)  then  into  the  NPS  Internet  using  CAT  5 
cabling. 

c.  Setup:  ITU  pre-recorded  wave  file  played  and  transmitted  from  the 
experiment  controller’s  desktop  to  the  test  subject’s  headset  using  Skype 
software. 

d.  Configuration:  A  proxy  server  is  interposed  between  the  test  subject  and  the 
Internet,  via  NPS  Internet.  Figure  2  shows  that  Station  Two  is  connected  to 
the  Internet  using  a  proxy  server,  Dell  Desktop  CPU  (Dimension  8200),  and 
Skype  software  installed  on  the  Dell  Inspiron  Notebook.  Prior  to  testing,  the 
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experiment  controller  ensured  the  proxy  server,  and  laptop  were  energized  and 
logged  onto  the  Internet.  The  experiment  controller  opened  and  activated  the 
Skype  account  for  “Non-MLS  Two”  client.  After  logging  onto  to  the  Skype 
Server,  the  experiment  controller  initiated  the  call  from  the  Dell  desktop  via 
the  Skype  Server.  After  the  call  was  initiated  and  received  by  the  “Non-MLS 
Two”  client,  connectivity  was  established.  Each  test  subject  listened  to  File 


Figure  2.  Station  Two 


7.  Station  Three 

a.  Equipment:  Dell  Inspiron  Notebook  CPU  (5150);  Plantronics  PC  Headset. 

b.  Network:  Dell  Desktop  CPU  (Dimension  8200)  physically  to  the  NPS 
Internet.  The  Dell  Inspiron  Notebook  CPU  (5150)  is  connected  to  a  Trusted 
Path  Extension  (TPE)  through  a  MYSEA  Server  (BAE  XTS  400)  then  into  the 
NPS  Internet  using  CAT  5  cabling. 

c.  Setup:  ITU  pre-recorded  wave  file  played  and  transmitted  from  the 
experiment  controller’s  desktop  to  the  test  subject’s  headset  using  Skype 
software. 
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d.  Configuration:  MYSEA  Server  and  Trusted  Path  Extension  (TPE)  are  routed 
between  Station  Three  and  the  Internet  via  NPS  Internet.  Figure  3  shows  that 
Station  Three  is  connected  to  the  Internet  via  MYSEA  using  Skype  software 
installed  on  the  Dell  Inspiron  Notebook.  Prior  to  testing,  the  experiment 
controller  ensured  the  MYSEA  server,  and  the  laptop  were  energized  and 
logged  onto  the  Internet.  The  experiment  controller  opened  and  activated  the 
Skype  account  for  “MLS  Three”  client.  After  logging  onto  to  the  Skype 
Server,  the  experiment  controller  initiated  the  call  from  the  Dell  desktop  via 
the  Skype  Server.  After  the  call  was  initiated  and  received  by  the  “MLS 
Three”  client,  connectivity  was  established.  Each  test  subject  listened  to  File 
“C”  at  Station  Three,  by  using  a  Plantronics  PC  Headset. 


Figure  3.  Station  Three 


8.  Station  Four 

a.  Equipment:  Dell  Inspiron  Notebook  CPU  (5150);  Plantronics  PC  Headset. 

b.  Network:  Dell  Desktop  CPU  (Dimension  8200)  physically  to  the  NPS 
Internet.  The  Dell  Inspiron  Notebook  CPU  (5150)  is  connected  to  a  Trusted 
Path  Extension  (TPE),  Dynamic  Security  Services  Servers  (Dell  Dimension 
8200),  and  the  MYSEA  Server  (BAE  XTS  400)  and  then  into  the  NPS 
Internet  using  CAT  5  cabling. 
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c.  Setup:  ITU  pre-recorded  wave  file  played  and  transmitted  from  the 
experiment  controller’s  desktop  to  the  test  subject’s  headset  using  Skype 
software. 

d.  Configuration:  MYSEA,  Dynamic  Security  Services  (DSS)/Trusted  Path 
Extension  (TPE)  Gateway  Servers  and  Trusted  Path  Extension  (TPE)  are 
routed  between  Station  Four  and  the  Internet  via  NPS  Internet.  Figure  4 
indicates  Station  Four  is  connected  to  the  Internet  via  MYSEA  using  Skype 
software  installed  on  the  Dell  Inspiron  Notebook.  Prior  to  testing,  the 
experiment  controller  ensured  the  MYSEA  server  and  the  laptop  were 
energized  and  logged  onto  the  Internet.  The  experiment  controller  opened  and 
activated  the  Skype  account  for  “MLS  Four”  client.  After  logging  onto  to  the 
Skype  Server,  the  experiment  controller  initiated  the  call  from  the  Dell 
desktop  via  the  Skype  Server.  After  the  call  was  initiated  and  received  by  the 
“MLS  Four”  client,  connectivity  was  established.  Each  test  subject  listened  to 


Figure  4.  Station  Four 
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9.  Testing  Procedures 

a.  Speech  Material:  The  ITU  sound  files  consisted  of  short  speech  passages, 
chosen  at  random  from  the  ITU  database  of  sample  sound  files  which  were 
simple  and  self-contained  in  meaning.  Each  sound  file  (segment)  has  a 
duration  of  not  less  than  9  seconds  and  not  more  than  15  seconds,  and 
consists  of  two  “sentences”,  which  are  separated  by  a  pause  of 
approximately  1  to  2  seconds.  All  sound  files  are  of  speech  with  a  British 
accent,  as  recommended  by  ITU-T,  and  contained  the  phrases  as  shown  in 
Appendix  B.  All  sound  files  were  played  for  each  test  subject  over  the 
Experiment  Controller’s  Dell  desktop  via  the  RealPlayer  Enterprise  Player 
and  transmitted  over  a  pair  of  external  computer  speakers  prior  to 
commencing  the  testing.  This  procedure  was  performed  to  familiarize  the 
test  subjects  with  the  accent,  gender,  configuration  and  length  of  each 
sound  file.  Additionally,  each  test  subject  was  allotted  ample  time  to 
review  the  questionnaire  and  become  prepared  to  answer  each  question  as 
thoroughly  and  completely  as  possible.  No  time  limits  were  imposed  for 
completion  of  each  of  the  questionnaires. 

b.  Listening  scale:  The  test  subjects  were  instructed  not  to  focus  on  the 
contents  of  the  sentences  but  instead  on  the  quality  of  sound.  The  rationale 
for  this  instruction,  which  was  not  conveyed  to  the  participants,  was  that 
diversions  regarding  other  aspects  of  the  sound  file  could  reduce  the 
quality  of  the  gathered  information  from  the  questionnaires,  resulting  in 
spurious  assessment  scores. 

c.  Procedures:  A  ten-item  questionnaire  was  distributed  and  available  at  each 
station.  Each  test  subject  was  afforded  ample  time  to  answer  the 
questionnaire  after  listening  to  the  recording  one  time.  Figure  5  shows  that 
each  test  subject  rotated  to  the  next  station  after  completing  their  assigned 
station. 
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d.  Period:  Experimental  sessions  lasted  from  30-35  minutes  for  completion 
of  all  four  stations.  Four  test  subjects  simultaneously  occupied  each  of  the 
four  stations  and  rotated  through  all  four  stations. 

(1)  Station  transition:  Approximately  1-2  minutes  per  test  subject,  per 

station;  total  transition  time  between  stations  for  all  test 
subjects  is  4-8  minutes. 

(2)  Sound  fde  transmission:  approximately  1  minute  per  station;  total 

sound  file  transmission  time  for  all  four  stations  is  4  minutes. 

(3)  Questionnaire  completion  time:  5-6  minutes  per  subject  for  the 

questionnaire  at  each  station,  which  resulted  in  a  20-25  minute 
period  for  completing  four  questionnaires. 


Figure  5.  Station  Configuration  &  Rotation 
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B.  EXPERIMENT  II 

The  objective  of  this  experiment  was  to  vary  the  order  in  which  each  of  the  four 
sound  files  were  presented  to  the  listener  to  determine  if  any  sound  files  were  of 
distinctly  different  quality  than  the  others  used  in  Experiment  I.  This  experiment  used  the 
same  standard  sound  files  used  in  Experiment  I.  The  sound  files  were  transmitted  to  two 
separate  test  stations  configured  exactly  the  same  using  Skype  software.33  The  two  test 
configurations  did  not  include  components  of  the  MYSEA  infrastructure,  and  used  the 
same  configuration  as  Station  One,  Experiment  I. 

1.  Subject  Sampling 

The  test  subjects  were  to  listen  and  judge  the  four  sound  files  in  a  pre-determined 
order,  not  known  by  the  test  subject,  over  two  identically  configured  stations  over 
identically  configured  networks  and  complete  four  separate  ten-item  questionnaires  for 
each  of  the  sound  files. 

All  test  subjects  were  random  volunteers  from  the  students  and  staff  associated 
with  the  Naval  Postgraduate  School.  The  sample  size  of  40  test  subjects  was  obtained 
from  a  total  population  of  1500  staff  and  students.34  For  a  confidence  level  of  95%  this 
sample  size  corresponds  to  a  15%  confidence  interval.35 

Subjects  were  solicited  through  on-campus  announcements  which  were  posted  at 
various  academic  buildings,  email  solicitations  from  academic  advisors/professors  and 
word  of  mouth  by  classmates.  These  methods  were  the  basis  of  success  in  obtaining  the 
required  sample  size  for  both  experiments. 

2.  Test  Question  Selection 

Testing  methods  were  centered  on  the  objective  of  obtaining  a  subjective 
performance  assessment  of  the  quality  of  sound  of  four  separate  sound  files.  The  method 


33  ITU-T  Recommendation  P.862  (2001)  -  Amendment  2. 

34  Fred  N.  Kerlinger,  Foundations  of  Behavioral  Research,  3rd  Edition,  (CBS  College  Publishing, 
1986),  117-119. 

35  Creative  Research  Systems,  The  Survey  System, 

http ://www/surveysy stem. com/sscalc.htm#ssneeded,  [Accessed  February  20,  2008] 
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was  a  “listening  test”  for  sound  file  quality,  where  messages  were  presented  aurally  to  all 
test  subjects.  Each  test  subject  was  given  a  ten-item  questionnaire  to  answer  after  each 
sound  file  was  played.  Each  of  the  test  subject’s  opinion  scores  were  rated  on  a 
cardinality  rating  scale  of  0  to  5,  with  5  being  of  greater  quality  (Appendix  A).  The 
cardinality  point  system  was  not  known  to  the  subject.  These  results  were  measures  of  the 
perceived  quality  in  four  categories  of  perception:  Listening-quality  effort;  Articulation; 
Acceptance  and  Overall  Impression.  These  areas  made  it  possible  to  compare  the 
effectiveness  of  different  speech  fdes  over  each  of  the  two  network  configurations.  The 
questionnaires  used  throughout  both  experiments  (I  and  II)  were  the  same  and  are 
included  in  Appendix  A.36 

3.  Sound  File  Selection 

ITU-T  Recommendation  P.862.3  was  used  to  provide  a  guide  for  estimating 
listening  speech  quality  by  using  reference  sound  files.  This  recommendation  provides 
the  necessary  information  for  obtaining  a  stable  and  reliable  reference  for  the  listener’s 
perception  of  speech  quality. 

It  was  recommended  that  the  speech  material  consist  of  simple,  short  and 
meaningful  sentences.  These  sentences  were  chosen  to  be  easy  to  understand  and  were 
constructed  into  sets  of  two  short  sentences  with  no  obvious  connection  of  meaning 
between  the  sentences  in  a  set.  Additionally,  in  accordance  with  P.830,  it  was 
recommended  that  a  minimum  of  two  female  and  two  male  sound  files  be  used  for 
testing.37 

4.  Controlling  Station 

The  experiment  controller  played  the  sample  files  for  all  test  subjects  prior  to 
testing  at  each  station  for  sound  and  accent  familiarity.  Additionally,  he  initiated  all 


36  ITU-T  Recommendation  P.85,  Telephone  Transmission  Quality  Subjective  Opinion  Tests,  June 
1994. 

37  ITU-T  Recommendation  P.862.3,  Telephone  Transmission  Quality,  Telephone  Installations,  Local 
Line  Networks,  November  2005. 
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telephone  connections  between  the  controlling  station  and  each  of  the  testing  stations  via 
Skype  VoIP  software.  The  experiment  controller  used  the  same  equipment  for  both 
Experiments  I  and  II. 

a.  Equipment:  Dell  Desktop  CPU  (Dimension  8200);  four  individual  ITU  pre¬ 
recorded  voice  sound  files,  (two  male,  two  female)  played  via  RealPlayer 
Enterprise  Player,  Build:  6.0.11.1526,  Copyright  (c)  2004;  (1)  4  ft,  3.5  mm 
stereo  audio  cable  (Male/Male). 

b.  Network:  Dell  Desktop  CPU  connected  to  the  Skype  server  via  Naval 
Postgraduate  School  Internet  (unclassified  Internet). 

c.  Setup:  A  standard  3.5mm  stereo  audio  cable  (Male/Male)  was  connected  to 
the  speaker  output  jack  (back  of  PC)  and  to  the  microphone  input  jack  (back 
of  PC)  to  create  a  loopback  circuitry  in  order  for  the  sound  file  to  be 
transmitted  to  the  Skype  Server  and  back  to  the  connected  client. 

NOTE:  This  testing  was  targeted  for  single  connectivity,  therefore,  no  conference  calling 
features  were  tested  or  evaluated.  All  calls  were  made  to  one  specific  station  at  a  time  and 


Figure  6.  Test  Station  One  (TS1)  and  Test  Station  Two  (TS2) 
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5.  Test  Station  One  (TS1)  and  Test  Station  Two  (TS2) 

a.  Equipment:  Dell  Inspiron  Notebook  CPU  (5150);  Plantronics  PC  Headset. 

b.  Network:  Dell  Desktop  CPU  (Dimension  8200)  physically  connected  to  the 
NPS  Internet  using  CAT  5  cabling. 

c.  Setup:  ITU  pre-recorded  wave  file  played  and  transmitted  from  the 
experiment  controller’s  desktop  to  the  test  subject’s  headset  using  Skype 
software. 

d.  Configuration:  As  shown  in  Figures  6,  Test  Stations  One  and  Two  are 
connected  directly  to  the  Internet,  via  NPS  Internet,  using  Skype  software 
installed  on  the  Dell  Inspiron  Notebook.  Prior  to  testing,  the  experiment 
controller  ensured  that  the  laptop  was  logged  on  and  connected  to  the  Internet. 
After  the  experiment  controller  logged  both  test  stations  onto  to  the  Skype 
Server,  he  initiated  the  call  from  the  Dell  desktop  via  the  Skype  Server.  After 
the  call  was  initiated,  the  “TS1”  and  “TS2”  clients  obtained  connectivity. 
Various  test  subjects  listened  to  the  four  sound  files  at  Test  Station  One,  by 
using  a  Plantronics  PC  Headset.  The  order  in  which  each  file  was  transmitted 
and  played  will  be  discussed  in  the  “Testing  Procedures”. 

6.  Testing  Procedures 

a.  Speech  Material:  The  ITU  sound  files  consisted  of  short  speech  passages, 
chosen  at  random  from  the  ITU  database  of  sample  sound  files  and  of  which 
were  simple  and  self-contained  in  meaning.  Each  sound  file  (segment)  has  a 
duration  of  not  less  than  9  seconds  and  not  more  than  15  seconds,  and  consists 
of  two  “sentences”,  which  are  separated  by  a  pause  of  approximately  1  to  2 
seconds.  All  sound  files  are  of  speech  with  a  British  accent,  as  recommended 
by  ITU-T,  as  shown  in  Appendix  B. 

b.  Configuration:  All  sound  files  were  played  for  each  test  subject  over  the 
Experiment  Controller’s  Dell  desktop  via  the  RealPlayer  Enterprise  Player 
and  transmitted  over  a  pair  of  external  computer  speakers  prior  to 
commencing  the  testing.  This  procedure  was  performed  to  familiarize  the  test 
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subjects  with  the  accent,  gender,  configuration  and  length  of  each  sound  file. 
Additionally,  each  test  subject  was  given  ample  time  in  reviewing  the 
questionnaire  in  order  to  become  prepared  to  answer  the  questions.  No  time 
limits  were  imposed  on  completing  each  of  the  questionnaires. 

c.  Listening  scale:  The  test  subjects  were  instructed  not  to  focus  on  the  contents 
of  the  sentences  but  instead  on  the  quality  of  sound.  The  rational  for  this 
instruction,  which  was  not  conveyed  to  the  participants,  was  that  diversions 
regarding  other  aspects  of  the  sound  file  could  reduce  the  quality  of  the 
gathered  information  from  the  questionnaires,  resulting  in  spurious  assessment 
scores. 

d.  Procedures:  Four  (4)  ten-item  questionnaires  were  distributed  and  available  at 
each  of  the  two  test  stations.  Each  test  subject  was  afforded  ample  time  to 
answer  the  questionnaire  after  listening  to  each  sound  file.  After  completing 
the  questionnaire  for  each  sound  file,  the  test  subject  indicated  to  the 
experiment  controller  that  he/she  was  ready  for  the  next  sound  file  to  be 
played.  Sound  files  were  played  in  a  fixed  order,  as  indicated  in  each  “Trial” 
configuration.  In  Experiment  II  test  subjects  were  tested  on  TS1  and  TS2, 
simultaneously. 

e.  Period:  Experimental  sessions  lasted  from  25-30  minutes  for  completion  of 
all  four  sound  files.  Two  test  subjects  simultaneously  occupied  each  of  the 
two  stations  during  playing  of  all  four  sound  files  for  each  Trial. 

f.  Sound  file  transmission:  approximately  1  minute  per  station;  total  sound  file 
transmission  time:  1  minute  per  station,  total  sound  file  transmission  time  for 
both  stations  were  2  minutes. 

g.  Questionnaire  completion  time:  5-6  minutes  per  subject  for  the  questionnaire 
at  each  station,  which  resulted  in  a  20-25  minute  period  for  completing  four 
questionnaires. 

This  experiment  subjectively  tested  voice  quality  of  Voice  over  Internet  Protocol 
on  four  network  configurations,  using  human  test  subjects.  The  outcome  of  Experiment  I 
produced  results  that  were  unanticipated  and  required  further  testing  to  validate  the 
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quality  of  each  of  the  sound  files.  The  ITU-T  sound  files  needed  to  be  evaluated  by  40 
test  subjects  not  exposed  to  Experiment  I.  The  conclusion  of  the  Experiment  II  indicated 
that  sound  file  “B”  difference  in  quality  from  the  other  sound  files. 
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IV.  VOICE  QUALITY  ANALYSIS 


A.  SCOPE  OF  ANALYSIS 

1.  Introduction 

This  chapter  presents  the  results  of  testing  used  to  evaluate  the  audio  quality  of 
VoIP  in  the  Monterey  Secure  Architecture  (MYSEA).  Single  Attribute  analysis  is  the 
primary  analysis  technique.  This  form  of  analysis  consists  of  sampling  a  single  attribute, 
i.e.,  voice  quality,  from  the  Mean  Opinion  Scores  (MOS)  derived  from  the  ten-item 
questionnaire  taken  from  subjective  testing  on  four  network  configurations..38 
Additionally,  “Multi-attribute  analysis”  which  is  a  comparison  of  MOS  results  of  two 
questions  was  used  in  the  analysis  of  Experiment  I. 

In  voice  communication,  audio  quality  usually  dictates  whether  a  listener’s 
experience  is  either  good  or  bad.  A  numerical  method  of  expressing  voice  quality  is 
required.  The  method  used  here  is  called  the  Mean  Opinion  Score  (MOS).  In  this  method, 
voice  quality  is  measured  by  having  randomly  selected  test  subjects  rate  the  quality  of 
test  sentences  transmitted  over  a  communications  circuit.39 

The  Mean  Opinion  Score  (MOS)  technique  accurately  determines  subjective 
results40.  For  this  experiment,  subjective  testing  required  randomly  selected  test  subjects 
to  listen  to  pre-recorded  standard  files,  and  have  them  rate  the  audio  quality  according  to 
their  perception.  To  rate  the  audio  quality,  the  test  subjects  were  provided  with  a  ten-item 
questionnaire.  The  questionnaire  answers  are  presented  in  Appendixes  C,  and  D. 

The  ten-item  questionnaire  conformed  to  a  Mean  Opinion  Score  by  rating  each 
response  with  a  numerical  value  ranging  from  0  to  5.  A  mapping  of  question  answers  to 
numerical  scores  is  shown  in  Appendix  A. 


38  Jay  L.  Devore,  Probability  and  Statistics  For  Engineering  and  the  Sciences  6th  Edition, 
Brooks/Cole,  2004,  713. 

39  ITU-T  Recommendation  P.862  (2001)  -  Amendment  2. 

40  William  C.  Hardy,  “VoIP  Service  Quality:  Measuring  and  Evaluating  Packet-switched  Voice”, 
McGraw-Hill  Professional,  2003,  151. 
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At  the  conclusion  of  Experiment  I,  the  results  from  one  of  the  stations  appeared 
anomalous.  To  identify  any  bias  resulting  from  the  quality  of  the  sound  fde,  all  sound 
fdes  used  in  Experiment  I  were  tested  for  relative  sound  quality  in  Experiment  II.  This 
evaluation  was  performed  using  the  same  process  as  in  Experiment  I. 

2.  Analysis  of  Experiment  I 

Figures  1  through  6  in  Chapter  III  present  a  pictorial  representation  of  both 
experiments  with  four  separate  test  configurations  for  Experiment  I,  and  one 
configuration  for  Experiment  II.  The  results  were  used  for  qualitative  analysis  of  Voice 
over  Internet  Protocol  (VoIP)  in  the  MYSEA  (Monterey  Security  Architecture)  test  bed 
using  Skype,  which  is  a  commercial  VoIP  client  and  server  software  product.  In 
Experiment  I,  each  configuration  is  a  separate  station  to  be  visited  by  each  test  subject. 
Each  station  added  a  component  to  the  configuration  for  which  voice  quality  in  the  test 
architecture  was  assessed.  Station  1  and  Station  2  measured  the  voice  quality  of  VoIP  in 
non-MYSEA  networks,  whereas,  Stations  3  &  4  incorporated  additional  equipment 
(variables)  from  the  MYSEA  test  bed  network.  A  different  sound  file  was  selected  to  be 
transmitted  to  each  of  the  test  stations.  (Appendix  B  identifies  each  sound  file  used  at 
each  specific  station). 

Completed  questionnaires  were  collected  and  entered  into  an  Excel  spreadsheet 
for  analysis  by  station  and  question,  (see  Tables  3-14  in  Appendix  C).  These  test 
questions  were  assigned  the  MOS  scores  (see  Appendix  A)  and  averaged  to  provide  an 
overall  MOS  rating  for  each  of  the  stations.  As  Chart  1  of  Appendix  C  shows,  Station 
Two  displays  a  higher  MOS  rating  than  that  the  other  three  stations.  This  result  was 
unexplainable  based  on  the  network  configuration  of  Station  Two. 

Because  Station  One’s  network  configuration  (see  Experimental  Design  Chapter, 
Figure  1)  was  the  simplest,  it  was  expected  to  have  presented  the  best  MOS  score.  Station 
Two’s  configuration,  (see  Experimental  Design  Chapter,  Figure  2),  included  a  “Proxy 
Server”  and  should  have  resulted  in  the  same  or  decreased  quality;  instead,  it  displayed 
increased  quality.  Stations  Three  and  Four  provided  MOS  scores  similar  in  perceived 
quality  to  Station  One.  Since  a  given  sound  file  was  always  presented  at  a  given  station,  it 
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was  suspected  that  the  sound  file  played  at  Station  Two  might  be  the  cause  of  the 
observed  anomaly.  Experiment  II  was  designed  to  examine  this  possibility. 

a.  Multi-attribute  Analysis  of  Experiment  I 

Multi-attribute  data  analysis  methods  complement  the  single-attribute 
analysis.  This  process  has  been  applied  to  the  results  of  Experiment  I  in  comparisons  of 
two  similar  questions.41  The  goal  of  this  analysis  is  to  ascertain  if  two  questions  of 
similar  meaning  provide  distinctive  results  when  subjectively  evaluating  the  quality  of 
voice  over  internet  protocol  (VoIP)  in  a  multilevel  secure  network. 

The  following  areas  were  assessed: 

1}  Articulation:  Perception  on  how  clear  the  pronunciation  of  each 
spoken  word  (distinguish  between  words). 

2)  Acceptance:  Indicates  if  the  voice  quality  was  found  to  be 
acceptable  for  telephone  conversations. 

3)  Listening  effort:  The  amount  of  effort  required  to  understand  the 
message. 

4)  Overall  impression:  Overall  assessment  of  word  clarity,  system 
connection  and  background  noise  when  listening  to  a  telephone  message. 

Figures  4  through  15  (see  Appendix  E),  are  divided  into  four  quadrants, 
representing  the  comparison  between  the  two  questions.  Quadrant  I  (upper  right  comer), 
is  represented  with  green  color  signifying  “good”  sound  quality;  Quadrants  II,  IV  (lower 
right  and  upper  left,  respectively)  are  yellow  signifying  “neutral”  sound  quality,  and 
Quadrant  III  (lower  left)  is  pink  signifying  “bad”  or  “poor”  sound  perception  of  the  test 
subjects  responses. 

This  methodology  of  analysis  reinforces  the  single-attribute  analysis  of  the 
questions  asked  in  reference  to  voice  quality,  sound  quality  and  word  clarity.  Utilizing 
multi-attribute  by  comparing  two  similar  questions  and  plotting  the  results  into  a 
quadratic  graph,  illustrates  how  the  test  subjects  perceived  quality  of  each  of  the  sound 
files  at  the  various  stations. 

41  F.  Murtagh  and  A.  Heck,  Multivariate  Data  Analysis,  Kluwer,  Dordrecht,  1987 


29 


3.  Question  3  /  Question  4 

The  first  multi-attribute  analysis  of  two  questions: 
Question  3:  The  voice  on  the  recording  was : 


o  I  Don’t  remember  or  I’m  not  sure  =  0 

o  Fading  =  1 

o  Low  Volume  =  2 

o  Clear  =  3 


Question  4:  The  voice  of  the  person  on  the  recording  sounded:  Was  it  like  an 
echo/hollow,  fuzzy/unnatural  or  was  it  clear  and  understandable? 


o  I  Don’t  remember  or  I’m  not  sure  =  0 

o  Fuzzy/Unnatural  =  1 

o  Echo/Hollow  =  2 

o  Clear  and  Understandable  =  3 


4.  Question  9  /  Question  1 

The  second  multi-attribute  analysis  of  two  questions: 

Question  9:  How  would  you  rate  the  quality  of  the  sound  of  what  you  have  just 


heard? 

o  I  Don’t  remember  or  I’m  not  sure  =  0 

o  Bad  =  1 

o  Poor  =  2 

o  Fair  =  3 

o  Good  =  4 

o  Excellent  =  5 


Question  1:  Which  of  these  four  words  comes  closest  to  describing  the  quality  of 
the  connection  during  the  recording? 


o  Poor  =  1 

o  Fair  =  2 

o  Good  =  3 

o  Excellent  =  4 
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5. 


Question  10  /  Question  5 


The  third  and  final  multi-attribute  analysis  of  two  questions: 

Question  10:  Were  all  of  the  words  distinguishable? 
o  I  Don’t  remember  or  I’m  not  sure  =  0 

o  No,  Not  at  all  =  1 

o  No,  Not  very  clear  =  2 

o  Fairly  Clear  =  3 

o  Yes,  Clear  enough  =4 

o  Yes,  Very  Clear  =5 

Question  5:  Did  you  find  certain  words  hard  to  understand? 
o  I  Don’t  remember  or  I’m  not  sure  =  0 

o  Often  =  1 

o  Occasionally  =  2 

o  Rarely  =  3 

o  Never  =  4 
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Figure  7.  Multi-Attribute  Comparison  of  All  Question  Pairs 

The  Multi-attribute  analysis  supports  the  results  of  the  single  attribute  analysis  as 
indicated  in  Figure  7.  Station  Two  received  much  higher  quadrant  one  scores  than  any  of 
the  other  stations. 

6.  Analysis  of  Experiment  II 

The  objective  of  this  experiment  was  to  vary  the  order  in  which  the  four  sound 
files  were  presented  to  a  listener  with  a  fixed  network  configuration  to  determine  if  one 
sound  file  was  of  better  quality  than  the  others.  This  experiment  used  the  same  ITU-T 
standard  sound  files  as  were  used  in  Experiment  I.  The  sound  files  were  transmitted  to 
two  separate  test  stations  configured  exactly  the  same  using  Skype  software.42  Forty 
randomly  and  newly  selected  test  subjects  participated  in  Experiment  II.  Unlike 
Experiment  I,  each  questionnaire  distributed  requested  the  gender  of  the  Test  Subject. 
This  information  was  to  help  identify  any  listener/speaker  gender  bias. 

42  ITU-T  Recommendation  P.862  (2001)  -  Amendment  2. 
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Experiment  II  was  designed  to  rotate  the  order  in  which  each  the  sound  files  were 
presented: 

Trial  1:  A,  B,  C,  D 
Trial  2:  B,  C,  D,  A 
Trial  3:  C,  D,  A,  B 
Trial  4:  D,  A,  B,  C 

Appendix  D,  Chart  7  shows,  each  sound  file  tabulated  by  the  test  subject’s  gender 
in  order  to  determine  if  the  anomalous  scores  in  Experiment  I,  Station  Two,  could  have 
come  from  gender  biasing.  Since  the  effect  of  test  subject  gender  on  the  results  were 
negligible,  gender  biasing  was  determined  not  to  have  been  a  factor  in  the  anomalous 
outcome  for  Experiments  I  or  II.  Each  question  response  was  tabulated  in  Tables  16 
through  31.  These  responses  were  averaged  and  compared  as  displayed  in  Charts  9 
through  12  of  Appendix  D. 

Figure  8  shows  that  Station  Two  in  Experiment  I  exhibited  significantly  higher 
scores  than  the  other  three  stations.  Additional  testing  was  required  to  determine  if  sound 
file  biasing  was  the  cause  of  the  anomalous  reading. 


Figure  8.  Station  Comparison  with  Upper/Lower  Confidence  limits 
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Figure  9  confirms  that  the  anomalous  result  for  Experiment  I  resulted  from  sound 
fde  quality;  in  Experiment  II.  The  higher  sound  quality  of  File  B  was  consistent  with  the 
higher  sound  quality  measured  for  Station  Two  in  Experiment  I;  indicating  that  the 
increase  was  due  to  the  quality  of  the  sound  file  as  perceived  by  test  subjects  and  not 
because  of  the  network  configuration  of  Station  Two. 
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Figure  9.  Experiment  II  Sound  File  Comparison 
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V.  CONCLUSION 


This  thesis  explored  the  quality  of  Voice  over  Internet  Protocol  (VoIP)  when 
configured  within  a  multilevel  secure  network  and  compared  it  to  voice  quality  within  a 
non-multilevel  secure  network.  The  standards  of  the  Telecommunication  Standardization 
Sector  (ITU-T),  P.862.3,  were  used  for  testing  of  voice  quality  within  VoIP  signals  in  a 
controlled  and  isolated  multilevel  network.43.  The  Monterey  Secure  Architecture, 
MYSEA,  was  the  test  bed  multilevel  secure  network,  where  VoIP  software,  Skype,  was 
used  to  transmit  the  sound  files.  This  call  processing  software  was  used  for  all 
experiments  and  on  all  test  stations. 

The  test  subjects  consisted  of  staff  and  students  from  the  Naval  Postgraduate 
School.  Each  test  subject  was  instructed  on  Skype ’s  operational  capability  and 
functionality.  Additionally,  they  were  instructed  to  listen  to  the  four  sound  files  and 
complete  a  ten-item  questionnaire  for  each  sound  file  which  were  assigned  to  a  specific 
testing  station.  Experiment  I  was  configured  for  the  test  subjects  to  evaluate  the  sound 
files  at  each  of  four  network  configurations.  Each  station  questionnaire  was  graded  and 
tabulated  using  a  Mean  Opinion  Score  (MOS),  which  was  used  for  single  and  multi 
attribute  analysis.  Once  the  MOS  was  scored  and  plotted,  an  anomalous  result  appeared 
for  Station  Two.  Since  a  particular  sound  file  was  always  presented  at  a  particular  station, 
it  was  suspected  that  the  sound  file  played  at  Station  Two  might  be  the  cause  of  the 
observed  anomaly.  Experiment  II  was  designed  to  examine  the  possibility  of  sound  file 
bias. 

The  objective  of  Experiment  II  was  to  determine  if  one  sound  file  was  of  better 
quality  than  the  others.  This  was  achieved  by  varying  the  order  in  which  the  four  sound 
files  were  presented  to  each  listener  within  a  fixed  network  configuration.  Using  the  same 
ITU-T  standard  sound  files  as  were  used  in  Experiment  I,  all  sound  files  were  transmitted 
to  two  separate  test  stations  configured  exactly  the  same,  as  shown  in  Figure  6,  using 
Skype  software.44  Forty  test  subjects  participated  in  Experiment  II.  The  questionnaires 

43  ITU-T  Recommendation  P.862  (2001)  -  Amendment  2. 

44  ITU-T  Recommendation  P.862  (2001)  -  Amendment  2. 
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for  Experiment  II  requested  the  gender  of  each  Test  Subject.  This  information  was  to  help 
identify  any  listener/speaker  gender  bias.  Experiment  II  rotated  the  order  in  which  each  of 
the  sound  files  was  presented  as  outlined  in  the  Experimental  Design  Chapter. 

Test  subject  gender  showed  negligible  differences  on  the  results  of  Experiment  II. 
Thus  test  subject  gender  bias  was  dismissed  as  a  factor  in  the  anomalous  outcome  for 
Station  Two  in  Experiment  I.  Experiment  II  results  provided  evidence  that  the  sound  file 
“B”  had  scores  indicating  a  statistically  significant  anomalous  outcome. 

All  of  Station  Two  results  provided  in  Experiment  I  were  disregarded  and  Station 
One  was  used  as  the  baseline  for  the  remaining  testing  and  analysis.  The  experimental 
results  indicate  that  voice  quality  in  multilevel  secure  networks  scored  as  “good”  or 
“better”  voice  quality  when  compared  to  the  same  characteristics  in  a  non-multilevel 
secure  network. 
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APPENDIX  A.  QUESTIONNAIRE 


Appendix  A  lists  the  ten-item  questionnaire  administered  to  participants  in  both 
Experiments  (I  &  II)  after  they  listened  to  each  of  the  sound  files. 

Each  question  is  assigned  a  point  value  for  analysis  computation. 
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Table  1.  Ten  Item  Questionnaire 


QUESTIONS 

CARDINALITY  POI1 

TT  ASSIGNMENT 

0 

1 

2 

3 

4 

5 

1 

Which  of  these  four 
words  comes  closest  to 
describing  the  quality  of 
the  connection  during 
the  recording? 

POOR 

FAIR 

GOOD 

EXCELLENT 

2 

Did  you  have  difficulty 
in  listening  over  the 
connection 

YES 

NO 

3 

The  voice  on  the 
recording  was: 

I  DON'T 

REMEMBER  I’M 
NOT  SURE 

FADING 

LOW 

VOLUME 

CLEAR 

4 

The  voice  of  the  person 
on  the  recording 

sounded:  Was  it  like  an 
echo/hollow, 
fuzzy/unnatural  or  was 
it  clear  and 

understandable? 

I  DON'T 

REMEMBER  I’M 
NOT  SURE 

FUZZY  / 
UNNATURA 

L 

ECHO/ 

HOLLOW 

CLEAR  AND 
UNDERSTAND 
-ABLE 

5 

Did  you  find  certain 
words  hard  to 

understand? 

I  DON'T 

REMEMBER  I’M 
NOT  SURE 

OFTEN 

OCCASION 

ALLY 

RARELY 

NEVER 

6 

How  would  you 

describe  the  effort  you 
were  required  to  make 
in  order  to  understand 
the  message? 

NOT  COMPRE¬ 
HENSIBLE  WITH 
ANY  EFFORT 

EFFORT 

REQUIRED 

MODERAT 

E  EFFORT 
REQUIRED 

ATTENTION 

NECESSARY; 

NO 

APPRECIABLE 

EFFORT 

REQUIRED 

COMPLETELY 

COMPREHEN 

SIBLE; 

NO  EFFORT 
REQUIRED 
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Table  1. _ Ten  Item  Questionnaire  (cont) 


QUESTIONS 

CARDINALITY  POINT  ASSIGNMEE 

IT 

0 

1 

2 

3 

4 

5 

7 

Tell  me  if  you  noticed  any 
of  these  during  your 
session:  A  rushing  or 
hissing  sound;  a  frying 
and  /or  sizzling, 

crackling  sound;  or  a 
humming  or  buzzing 
sound? 

I  DON'T 

REMEMBER  I'M 
NOT  SURE 

HUMMING, 

BUZZING 

FRYING 

AND/OR 

SIZZLING, 

CRACKLING 

RUSHING, 

HISSING 

I  DIDN'T 
NOTICE  ANY 
NOISE  OR 
ANYTHING 
OTHER  THAN 
THE 

RECORDING 

8 

Please  try  to  remember 
the  background  noise  in 
the  area  around  your 
station  (e.g.,  noise  from 
the  air-conditioning  unit, 
office  equipment,  or 
other  people  talking) 
while  you  were  involved 
in  the  testing.  Which  of 
the  following  categories 
best  describes  your 
experience? 

I  DON'T 

REMEMBER  I'M 
NOT  SURE 

VERY 

NOISY 

NOISY 

QUIET 

VERY  QUIET 

9 

How  would  you  rate  the 
quality  of  the  sound  of 
what  you  have  just 
heard? 

I  DON'T 

REMEMBER  I'M 
NOT  SURE 

BAD 

POOR 

FAIR 

GOOD 

EXCELLENT 

10 

Were  all  of  the  words 
distinguishable? 

I  DON'T 

REMEMBER  I'M 
NOT  SURE 

NO,  NOT 
AT  ALL 

NO,  NOT 
VERY 
CLEAR 

FAIRLY  CLEAR 

YES,  CLEAR 
ENOUGH 

YES,  VERY 
CLEAR 
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APPENDIX  B.  SOUND  FILES 


The  International  Telecommunications  Union  Standardization  Sector  (ITU-T)  provided 
sound  fdes  were  used  in  end-to-end  speech  quality  assessment  of  telephone  networks. 

1.  Sound  fde  “A”:  British  female  voice  (afls03b2cl7)  with  the  words: 

“He  took  out  his  pipe  and  lit  up.  It  was  a  separate  bar.  ’’ 

2.  Sound  fde  “B”:  British  male  voice  (amls03b2c4)  with  the  words: 

“He  carried  a  bag  of  tennis  balls.  The  scheme  was  plotted  out.  ” 

3.  Sound  fde  “C”:  British  female  voice  (afls01b2cl6)  with  the  words: 

“You  were  the  perfect  Hostess.  Are  you  going  to  be  nice  to  me?  ” 

4.  Sound  file  “D”:  British  male  voice  (amls02b2c4)  with  the  words: 

“There  wasn ’t  a  scrap  of  evidence.  The  jar  was  full  of  water.  ” 
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APPENDIX  C.  EXPERIMENT  I  DATA 


Appendix  C  shows  questionnaire  data  from  each  of  the  61  test  subjects  taken  during  Experiment  I. 


Table  2.  Station  Average  MOS  Scores 


AVERAGE  DATA  COMPARISO 

N 

STATIONS/QUESTIONS 

Ql 

Q2 

Q3 

Q4 

Q5 

Q6 

Q7 

Q8 

Q9 

Q10 

STATION  ONE  AVERAGE  (SI) 

2.70 

2.31 

2.30 

1.93 

2.25 

2.20 

3.05 

2.51 

3.59 

3.10 

STATION  TWO  AVERAGE  (S2) 

3.52 

2.93 

2.97 

2.79 

3.36 

3.38 

3.64 

2.64 

4.30 

4.20 

STATION  THREE  AVERAGE  (S3) 

2.82 

2.31 

2.49 

2.00 

2.30 

2.36 

3.23 

2.44 

3.67 

3.10 

STATION  FOUR  AVERAGE  (S4) 

2.85 

2.41 

2.64 

2.15 

2.54 

2.43 

3.07 

2.48 

3.62 

3.25 

Table  2  is  the  average  scores  for  each  question  for  each  station.  See  Appendix  A  for  Mean  Opinion  Score  (MOS)  point  assignment  to 
questionnaire  complete  sentences. 
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Chart  1.  Station  Comparison  of  All  Questions 

Chart  1  compares  the  average  score  for  all  question  for  each  station  . 
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Chart  2.  Stations  and  Question  Comparison 


Chart  2  shows  the  comparison  of  each  station  by  each  individual  question.  The  Mean  Opinion  Score  value  assigned  is  detailed  in  the 
information  box  in  the  upper  right  comer  of  Chart  2. 
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Table  3.  Station  One  Raw  Data 


TEST  STATION 
One 

SURVEY  QUESTIONS 

Qi 

02 

03 

04 

Os 

06 

Q7 

Q8 

Q9 

Q10 

Test  Subject  1 

2 

3 

1 

2 

2 

3 

2 

2 

3 

4 

Test  Subject  2 

2 

1 

3 

1 

2 

1 

3 

2 

3 

2 

Test  Subject  3 

1 

1 

2 

1 

1 

0 

3 

2 

2 

1 

Test  Subject  4 

4 

3 

3 

3 

2 

1 

4 

2 

5 

2 

Test  Subject  5 

4 

3 

3 

3 

1 

1 

4 

2 

5 

1 

Test  Subject  6 

1 

1 

1 

1 

1 

1 

3 

4 

3 

2 

Test  Subject  7 

2 

3 

3 

1 

2 

2 

1 

2 

3 

3 

Test  Subject  8 

3 

3 

1 

1 

2 

3 

4 

2 

3 

3 

Test  Subject  9 

4 

3 

3 

3 

3 

3 

4 

2 

5 

4 

Test  Subject  10 

2 

1 

3 

1 

2 

2 

4 

2 

3 

3 

Test  Subject  11 

3 

3 

3 

3 

3 

4 

4 

2 

4 

4 

Test  Subject  12 

4 

3 

3 

3 

3 

3 

4 

3 

4 

4 

Test  Subject  13 

1 

1 

1 

1 

1 

0 

0 

3 

3 

1 

Test  Subject  14 

3 

3 

3 

3 

2 

1 

4 

2 

4 

4 

Test  Subject  15 

1 

1 

2 

1 

1 

0 

0 

3 

2 

1 

Test  Subject  16 

3 

3 

3 

3 

2 

2 

0 

2 

4 

3 

Test  Subject  17 

2 

1 

2 

1 

2 

3 

4 

2 

3 

3 

Test  Subject  18 

2 

1 

1 

1 

1 

1 

2 

2 

2 

2 

Test  Subject  19 

4 

3 

3 

3 

2 

4 

4 

4 

5 

5 

Test  Subject  20 

2 

1 

2 

1 

2 

1 

2 

3 

3 

3 
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Table  4.  Station  One  Raw  Data  (cont) 


TEST  STATION 
One 

SURVEY  < 

QUESTIONS 

Qi 

Q2 

q3 

q4 

Qs 

q6 

Q7 

Qs 

Q9 

Q10 

Test  Subject  21 

2 

3 

1 

2 

2 

2 

2 

2 

3 

2 

Test  Subject  22 

2 

3 

1 

1 

2 

3 

4 

3 

4 

3 

Test  Subject  23 

3 

3 

1 

3 

2 

3 

4 

3 

4 

3 

Test  Subject  24 

3 

1 

1 

1 

2 

3 

4 

4 

4 

3 

Test  Subject  25 

3 

3 

3 

3 

2 

3 

4 

2 

4 

3 

Test  Subject  26 

3 

3 

3 

3 

3 

3 

4 

3 

4 

3 

Test  Subject  27 

2 

3 

1 

3 

4 

4 

2 

3 

3 

4 

Test  Subject  28 

2 

1 

2 

1 

1 

1 

2 

2 

2 

2 

Test  Subject  29 

4 

1 

3 

1 

3 

3 

4 

3 

4 

4 

Test  Subject  30 

4 

3 

3 

3 

2 

3 

4 

2 

4 

4 

Test  Subject  31 

3 

3 

3 

1 

2 

2 

4 

3 

3 

3 

Test  Subject  32 

2 

3 

3 

1 

4 

3 

4 

2 

3 

4 

Test  Subject  33 

3 

3 

2 

1 

4 

3 

1 

2 

4 

4 

Test  Subject  34 

2 

1 

0 

1 

2 

1 

0 

3 

2 

2 

Test  Subject  35 

4 

3 

3 

3 

3 

2 

4 

3 

5 

4 

Test  Subject  36 

3 

3 

3 

3 

2 

1 

4 

2 

4 

4 

Test  Subject  37 

2 

1 

3 

1 

2 

2 

2 

2 

3 

3 

Test  Subject  38 

4 

3 

3 

3 

4 

4 

3 

2 

5 

5 

Test  Subject  39 

3 

1 

3 

1 

2 

2 

4 

3 

3 

3 

Test  Subject  40 

3 

3 

1 

2 

2 

3 

4 

2 

4 

3 
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Table  5.  Station  One  Raw  Data  (cont) 


TEST  STATION 
One 

SURVEY  < 

QUESTIONS 

Oi 

Q2 

03 

04 

Os 

06 

Q7 

Q8 

Q9 

Q10 

Test  Subject  41 

3 

3 

3 

3 

4 

3 

2 

2 

4 

4 

Test  Subject  42 

4 

3 

3 

3 

3 

4 

4 

2 

5 

4 

Test  Subject  43 

3 

3 

3 

1 

3 

3 

4 

2 

3 

3 

Test  Subject  44 

3 

1 

1 

1 

2 

2 

3 

3 

3 

3 

Test  Subject  45 

3 

3 

3 

3 

2 

2 

4 

4 

4 

4 

Test  Subject  46 

2 

1 

3 

1 

1 

0 

2 

3 

3 

2 

Test  Subject  47 

3 

3 

3 

1 

1 

1 

4 

3 

4 

2 

Test  Subject  48 

1 

3 

3 

3 

1 

1 

4 

2 

5 

3 

Test  Subject  49 

3 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Test  Subject  50 

2 

1 

1 

1 

2 

1 

4 

2 

3 

3 

Test  Subject  51 

4 

3 

3 

3 

2 

3 

4 

2 

5 

3 

Test  Subject  52 

2 

3 

1 

1 

2 

1 

2 

2 

3 

3 

Test  Subject  53 

3 

3 

3 

3 

2 

1 

4 

3 

3 

3 

Test  Subject  54 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

Test  Subject  55 

2 

1 

1 

1 

2 

3 

3 

3 

3 

2 

Test  Subject  56 

3 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Test  Subject  57 

4 

3 

3 

3 

4 

4 

0 

3 

4 

4 

Test  Subject  58 

3 

3 

2 

1 

3 

2 

3 

3 

4 

4 

Test  Subject  59 

2 

3 

3 

3 

2 

3 

3 

3 

4 

3 

Test  Subject  60 

2 

1 

1 

1 

2 

1 

0 

3 

2 

3 

Test  Subject  61 

2 

1 

2 

1 

2 

2 

4 

2 

3 

2 

Tables  3,  4  and  5  show  the  responses  from  the  61  test  subjects  who  listened  to  Sound  File  “A”  at  Station  One. 
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STATION  ONE  COMPARISON 


AVERAGE 
SCORES  (61 
TEST 

S  INJECTS) 


Chart  3.  Station  One  Average  Question  Responses 

Chart  3  shows  the  average  scores  of  61  test  subjects  on  the  ten  item  questionnaire  at  Station  One. 
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Table  6.  Station  Two  Raw  Data 


TEST  STATION 
Two 

SURVEY  QUESTIONS 

Qi 

02 

q3 

q4 

Qs 

q6 

Q7 

Qs 

Q9 

Q10 

Test  Subject  1 

3 

3 

3 

2 

3 

2 

1 

2 

4 

4 

Test  Subject  2 

4 

3 

3 

3 

4 

4 

4 

2 

4 

5 

Test  Subject  3 

3 

3 

3 

3 

3 

2 

1 

2 

4 

3 

Test  Subject  4 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Test  Subject  5 

4 

3 

3 

3 

4 

3 

4 

2 

5 

4 

Test  Subject  6 

4 

3 

3 

3 

4 

4 

4 

4 

5 

5 

Test  Subject  7 

2 

3 

3 

2 

3 

3 

1 

3 

3 

4 

Test  Subject  8 

4 

3 

3 

3 

4 

4 

4 

2 

4 

4 

Test  Subject  9 

4 

3 

3 

3 

3 

3 

4 

2 

5 

4 

Test  Subject  10 

3 

3 

3 

1 

4 

3 

4 

2 

4 

4 

Test  Subject  11 

3 

3 

3 

0 

4 

3 

2 

1 

4 

4 

Test  Subject  12 

4 

3 

3 

3 

4 

4 

4 

4 

5 

5 

Test  Subject  13 

4 

3 

3 

3 

4 

4 

4 

3 

4 

5 

Test  Subject  14 

4 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Test  Subject  15 

3 

3 

3 

3 

2 

3 

4 

3 

4 

4 

Test  Subject  16 

4 

3 

3 

3 

2 

3 

4 

2 

4 

3 

Test  Subject  17 

4 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Test  Subject  18 

4 

3 

3 

3 

4 

3 

4 

4 

5 

3 

Test  Subject  19 

4 

3 

3 

3 

4 

4 

4 

4 

5 

5 

Test  Subject  20 

4 

3 

3 

3 

2 

3 

4 

3 

5 

4 
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Table  7.  Station  Two  Raw  Data  (cont) 


TEST  STATION 
Two 

SURVEY  QUESTIONS 

Qi 

Qi 

q3 

q4 

Qs 

q6 

Q7 

Qs 

Q9 

Q10 

Test  Subject  21 

3 

3 

3 

2 

2 

3 

3 

2 

4 

4 

Test  Subject  22 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

Test  Subject  23 

4 

3 

3 

3 

4 

4 

4 

4 

5 

5 

Test  Subject  24 

4 

3 

3 

3 

4 

4 

4 

4 

5 

5 

Test  Subject  25 

3 

3 

3 

3 

3 

4 

4 

2 

5 

4 

Test  Subject  26 

4 

3 

3 

3 

4 

3 

4 

4 

4 

4 

Test  Subject  27 

4 

3 

3 

3 

4 

4 

4 

4 

5 

5 

Test  Subject  28 

3 

3 

3 

3 

4 

4 

4 

2 

4 

4 

Test  Subject  29 

3 

1 

3 

3 

3 

3 

4 

3 

4 

4 

Test  Subject  30 

4 

3 

3 

3 

4 

4 

4 

2 

4 

5 

Test  Subject  31 

3 

3 

3 

3 

2 

3 

4 

3 

3 

3 

Test  Subject  32 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Test  Subject  33 

4 

3 

3 

3 

3 

3 

4 

2 

5 

4 

Test  Subject  34 

3 

3 

3 

3 

3 

4 

4 

3 

4 

4 

Test  Subject  35 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

Test  Subject  36 

4 

3 

3 

3 

4 

4 

4 

2 

5 

4 

Test  Subject  37 

3 

3 

3 

1 

3 

3 

2 

2 

3 

3 

Test  Subject  38 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Test  Subject  39 

3 

3 

3 

3 

2 

3 

3 

3 

4 

3 

Test  Subject  40 

4 

3 

3 

3 

4 

4 

4 

2 

5 

4 
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Table  8.  Station  Two  Raw  Data  (cont) 


TEST  STATION 
Two 

SURVEY ( 

QUESTIONS 

Oi 

Q2 

Q3 

04 

Os 

06 

Q7 

Q8 

Q9 

Q10 

Test  Subject  41 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Test  Subject  42 

3 

3 

3 

3 

4 

4 

4 

2 

4 

4 

Test  Subject  43 

1 

1 

1 

1 

1 

1 

2 

2 

2 

2 

Test  Subject  44 

3 

3 

3 

3 

3 

3 

4 

4 

4 

4 

Test  Subject  45 

3 

3 

3 

3 

4 

3 

4 

4 

4 

5 

Test  Subject  46 

4 

3 

3 

3 

3 

4 

4 

3 

4 

4 

Test  Subject  47 

4 

3 

3 

3 

2 

4 

4 

3 

4 

4 

Test  Subject  48 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Test  Subject  49 

3 

3 

3 

3 

4 

4 

4 

2 

4 

5 

Test  Subject  50 

3 

3 

3 

3 

2 

2 

2 

2 

3 

3 

Test  Subject  51 

4 

3 

3 

3 

4 

3 

4 

2 

5 

5 

Test  Subject  52 

3 

3 

3 

3 

4 

3 

4 

2 

4 

4 

Test  Subject  53 

3 

3 

3 

3 

2 

2 

4 

3 

3 

4 

Test  Subject  54 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

Test  Subject  55 

3 

3 

3 

2 

2 

3 

4 

3 

4 

3 

Test  Subject  56 

4 

3 

3 

3 

4 

4 

4 

2 

4 

4 

Test  Subject  57 

3 

3 

3 

3 

3 

3 

3 

3 

4 

4 

Test  Subject  58 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

Test  Subject  59 

4 

3 

3 

3 

4 

4 

4 

4 

5 

5 

Test  Subject  60 

3 

3 

3 

3 

3 

3 

4 

3 

4 

4 

Test  Subject  61 

3 

3 

3 

3 

3 

2 

2 

2 

4 

4 

Tables  6,  7,  and  8  show  the  responses  from  the  61  test  subjects  who  listened  to  Sound  File  “B”  at  Station  Two. 
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/" 


AVERAGE 
SCORES  (61 

TEST 

SUBJECTS) 


Chart  4.  Station  Two  Average  Question  Responses 


Chart  4  shows  the  average  scores  of  61  test  subjects  on  the  ten  item  questionnaire  at  Station  Two. 
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Table  9.  Station  Three  Raw  Data 


TEST  STATION 
Three 

SURVEY  QUESTIONS 

Qi 

q2 

q3 

q4 

Qs 

Qe 

Q7 

Qs 

Q9 

Q10 

Test  Subjects  1 

2 

1 

1 

2 

2 

2 

1 

2 

2 

2 

Test  Subjects  2 

2 

3 

0 

1 

2 

1 

2 

2 

2 

2 

Test  Subjects  3 

2 

1 

3 

2 

1 

1 

4 

2 

4 

4 

Test  Subjects  4 

3 

3 

3 

3 

2 

1 

4 

2 

5 

2 

Test  Subjects  5 

4 

3 

3 

3 

4 

3 

4 

2 

5 

4 

Test  Subjects  6 

2 

1 

3 

1 

2 

2 

4 

4 

5 

3 

Test  Subjects  7 

2 

3 

3 

2 

2 

1 

1 

3 

2 

3 

Test  Subjects  8 

4 

3 

3 

0 

2 

3 

4 

2 

4 

4 

Test  Subjects  9 

3 

3 

3 

3 

2 

3 

4 

2 

4 

4 

Test  Subjects  10 

3 

3 

3 

1 

2 

2 

4 

2 

4 

3 

Test  Subjects  11 

2 

1 

3 

1 

2 

2 

2 

1 

3 

3 

Test  Subjects  12 

2 

1 

2 

1 

2 

2 

3 

2 

3 

3 

Test  Subjects  13 

2 

1 

3 

2 

2 

3 

2 

3 

3 

3 

Test  Subjects  14 

4 

3 

3 

3 

3 

3 

4 

2 

5 

4 

Test  Subjects  15 

2 

1 

0 

1 

1 

1 

3 

3 

3 

2 

Test  Subjects  16 

2 

1 

3 

1 

1 

1 

2 

2 

2 

1 

Test  Subjects  17 

3 

1 

3 

1 

2 

2 

4 

2 

4 

3 

Test  Subjects  18 

4 

3 

3 

3 

3 

4 

4 

3 

5 

4 

Test  Subjects  19 

3 

3 

3 

3 

2 

3 

4 

4 

4 

4 

Test  Subjects  20 

3 

3 

3 

2 

2 

3 

4 

3 

3 

3 
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Table  10.  Station  Three  Raw  Data  (cont) 


TEST  STATION 
Three 

SURVEY  QUESTIONS 

Qi 

02 

q3 

q4 

Qs 

06 

Q7 

Os 

O9 

Q10 

Test  Subjects  21 

3 

3 

1 

3 

2 

3 

2 

2 

4 

3 

Test  Subjects  22 

2 

1 

3 

1 

2 

3 

2 

3 

3 

3 

Test  Subjects  23 

2 

3 

2 

1 

3 

3 

2 

2 

3 

4 

Test  Subjects  24 

4 

3 

3 

3 

4 

4 

4 

4 

5 

5 

Test  Subjects  25 

3 

3 

3 

3 

3 

3 

4 

2 

5 

4 

Test  Subjects  26 

2 

3 

3 

1 

2 

3 

3 

2 

3 

3 

Test  Subjects  27 

3 

3 

3 

2 

4 

4 

3 

3 

4 

4 

Test  Subjects  28 

3 

3 

3 

3 

4 

4 

4 

2 

4 

4 

Test  Subjects  29 

3 

1 

3 

1 

2 

2 

4 

3 

3 

3 

Test  Subjects  30 

4 

3 

3 

3 

2 

3 

4 

2 

5 

4 

Test  Subjects  31 

4 

3 

3 

3 

4 

4 

4 

3 

4 

4 

Test  Subjects  32 

3 

3 

3 

3 

4 

4 

4 

2 

4 

5 

Test  Subjects  33 

3 

3 

3 

1 

3 

3 

1 

2 

4 

3 

Test  Subjects  34 

3 

3 

3 

2 

2 

1 

4 

3 

4 

2 

Test  Subjects  35 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

Test  Subjects  36 

3 

3 

3 

3 

2 

3 

4 

2 

3 

3 

Test  Subjects  37 

2 

3 

3 

2 

2 

2 

2 

2 

3 

3 

Test  Subjects  38 

4 

3 

3 

3 

3 

3 

4 

2 

5 

3 

Test  Subjects  39 

2 

1 

0 

1 

1 

1 

3 

3 

3 

2 

Test  Subjects  40 

3 

1 

1 

1 

2 

3 

3 

2 

3 

3 
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Table  11.  Station  Three  Raw  Data  (cont) 


TEST  STATION 
Three 

SURVEY  QUESTIONS 

Qi 

Q2 

Q3 

q4 

q5 

06 

Q7 

Q8 

Q9 

Q10 

Test  Subjects  41 

4 

3 

3 

3 

2 

2 

4 

2 

4 

2 

Test  Subjects  42 

2 

1 

1 

1 

2 

2 

3 

2 

3 

2 

Test  Subjects  43 

3 

1 

3 

2 

2 

2 

4 

2 

3 

3 

Test  Subjects  44 

2 

1 

3 

2 

2 

2 

3 

3 

3 

2 

Test  Subjects  45 

2 

3 

1 

1 

4 

3 

2 

2 

3 

4 

Test  Subjects  46 

2 

1 

1 

2 

2 

2 

4 

3 

4 

3 

Test  Subjects  47 

3 

3 

3 

3 

1 

0 

4 

3 

4 

2 

Test  Subjects  48 

4 

3 

3 

3 

1 

1 

4 

2 

5 

2 

Test  Subjects  49 

3 

3 

3 

3 

4 

3 

4 

2 

4 

3 

Test  Subjects  50 

2 

1 

1 

2 

1 

1 

3 

2 

2 

2 

Test  Subjects  51 

2 

1 

3 

2 

1 

0 

3 

2 

3 

2 

Test  Subjects  52 

2 

3 

1 

1 

1 

1 

4 

2 

3 

1 

Test  Subjects  53 

3 

3 

3 

3 

2 

2 

4 

3 

4 

3 

Test  Subjects  54 

3 

3 

3 

1 

2 

3 

4 

3 

4 

3 

Test  Subjects  55 

3 

3 

3 

2 

2 

2 

4 

3 

4 

3 

Test  Subjects  56 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Test  Subjects  57 

3 

3 

3 

3 

3 

3 

4 

3 

4 

4 

Test  Subjects  58 

4 

3 

3 

2 

2 

1 

3 

3 

4 

3 

Test  Subjects  59 

3 

3 

3 

1 

3 

4 

3 

3 

4 

4 

Test  Subjects  60 

2 

1 

1 

2 

2 

2 

0 

3 

3 

3 

Test  Subjects  61 

2 

1 

1 

1 

1 

1 

0 

2 

2 

2 

Tables  9,  10,  and  1 1  show  the  responses  from  the  61  test  subjects  who  listened  to  Sound  File  “C”  at  Station  Three. 
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STATION  THREE  COMPARISON 


r 


AVERAGE  SCORES  (61 
TEST  SUBJECTS) 


J 


Chart  5.  Stations  Three  Average  Question  Responses 

Chart  5  shows  the  average  scores  of  61  test  subjects  on  the  ten  item  questionnaire  at  Station  Three. 
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Table  12.  Station  Four  Raw  Data 


TEST  STATION 
Four 

SURVEY  QUESTIONS 

Qi 

Q2 

q3 

q4 

Qs 

q6 

Q7 

Qs 

Q9 

Q10 

Test  Subject  1 

3 

3 

3 

3 

4 

3 

4 

2 

5 

4 

Test  Subject  2 

3 

3 

3 

1 

3 

3 

4 

2 

4 

3 

Test  Subject  3 

3 

3 

3 

3 

3 

2 

4 

2 

4 

4 

Test  Subject  4 

4 

1 

3 

3 

2 

2 

2 

2 

3 

3 

Test  Subject  5 

4 

3 

3 

3 

3 

3 

4 

2 

5 

4 

Test  Subject  6 

3 

3 

3 

1 

3 

2 

2 

3 

5 

3 

Test  Subject  7 

1 

1 

1 

1 

1 

1 

1 

2 

1 

1 

Test  Subject  8 

3 

3 

1 

1 

2 

2 

4 

2 

3 

3 

Test  Subject  9 

3 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Test  Subject  10 

3 

3 

3 

2 

3 

3 

4 

2 

4 

3 

Test  Subject  11 

2 

1 

3 

1 

2 

2 

1 

1 

3 

2 

Test  Subject  12 

4 

3 

3 

3 

3 

3 

4 

4 

5 

5 

Test  Subject  13 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

Test  Subject  14 

4 

3 

3 

3 

3 

3 

4 

2 

5 

4 

Test  Subject  15 

3 

3 

3 

0 

3 

2 

1 

3 

3 

3 

Test  Subject  16 

2 

1 

3 

1 

1 

1 

2 

2 

2 

1 

Test  Subject  17 

3 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Test  Subject  18 

3 

1 

3 

3 

2 

3 

1 

3 

4 

2 

Test  Subject  19 

1 

1 

2 

1 

1 

0 

3 

3 

2 

2 

Test  Subject  20 

4 

3 

3 

3 

2 

4 

4 

3 

4 

4 
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Table  13.  Station  Four  Raw  Data  (cont) 


TEST  STATION 
Four 

SURVEY  QUESTIONS 

Qi 

Q2 

q3 

q4 

Qs 

q6 

Q7 

Qs 

Q9 

Q10 

Test  Subject  21 

3 

3 

3 

3 

2 

3 

4 

2 

4 

4 

Test  Subject  22 

2 

1 

1 

1 

2 

2 

3 

3 

3 

2 

Test  Subject  23 

3 

3 

3 

2 

2 

3 

3 

3 

4 

3 

Test  Subject  24 

3 

3 

3 

3 

3 

3 

4 

4 

4 

4 

Test  Subject  25 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Test  Subject  26 

3 

3 

3 

3 

3 

3 

4 

3 

4 

4 

Test  Subject  27 

4 

3 

3 

3 

4 

4 

3 

3 

5 

4 

Test  Subject  28 

3 

3 

3 

3 

3 

2 

4 

2 

4 

3 

Test  Subject  29 

3 

1 

3 

1 

2 

3 

4 

3 

4 

3 

Test  Subject  30 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Test  Subject  31 

3 

3 

3 

3 

3 

3 

4 

3 

4 

4 

Test  Subject  32 

3 

3 

3 

1 

4 

4 

4 

2 

4 

5 

Test  Subject  33 

2 

1 

3 

2 

2 

2 

3 

2 

3 

3 

Test  Subject  34 

2 

1 

1 

1 

2 

3 

0 

3 

3 

3 

Test  Subject  35 

4 

3 

3 

3 

4 

3 

4 

3 

5 

5 

Test  Subject  36 

3 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Test  Subject  37 

1 

3 

3 

1 

1 

1 

2 

2 

2 

2 

Test  Subject  38 

4 

3 

3 

2 

4 

4 

2 

2 

5 

5 

Test  Subject  39 

2 

1 

0 

1 

2 

2 

3 

3 

3 

3 

Test  Subject  40 

2 

1 

1 

1 

2 

2 

2 

2 

2 

2 
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Table  14.  Station  Four  Raw  Data  (cont) 


TEST  STATION 
Four 

SURVEY  QUESTIONS 

Qi 

Q2 

Q3 

04 

q5 

06 

Q7 

Q8 

Q9 

Q10 

Test  Subject  41 

2 

1 

1 

1 

1 

1 

1 

2 

2 

1 

Test  Subject  42 

3 

3 

3 

3 

3 

3 

3 

2 

4 

3 

Test  Subject  43 

1 

1 

3 

2 

1 

1 

4 

2 

1 

2 

Test  Subject  44 

2 

1 

3 

1 

1 

1 

3 

3 

2 

1 

Test  Subject  45 

3 

3 

3 

2 

4 

3 

0 

3 

3 

4 

Test  Subject  46 

4 

3 

3 

3 

3 

3 

0 

3 

5 

4 

Test  Subject  47 

3 

3 

3 

3 

1 

1 

2 

3 

4 

2 

Test  Subject  48 

3 

3 

3 

3 

2 

2 

4 

2 

4 

3 

Test  Subject  49 

3 

3 

3 

3 

4 

4 

4 

2 

4 

4 

Test  Subject  50 

3 

3 

2 

3 

3 

2 

4 

2 

3 

3 

Test  Subject  51 

3 

1 

3 

1 

2 

1 

3 

2 

3 

3 

Test  Subject  52 

2 

3 

1 

2 

2 

1 

3 

2 

3 

2 

Test  Subject  53 

3 

3 

3 

3 

2 

2 

4 

3 

4 

3 

Test  Subject  54 

3 

3 

3 

3 

2 

3 

4 

3 

4 

4 

Test  Subject  55 

3 

3 

3 

2 

2 

3 

4 

3 

4 

4 

Test  Subject  56 

3 

3 

3 

3 

4 

3 

4 

2 

4 

4 

Test  Subject  57 

1 

1 

1 

1 

1 

0 

2 

2 

1 

1 

Test  Subject  58 

3 

3 

3 

1 

1 

0 

1 

3 

4 

2 

Test  Subject  59 

4 

3 

3 

3 

4 

4 

4 

4 

5 

5 

Test  Subject  60 

2 

3 

3 

2 

3 

2 

4 

3 

3 

4 

Test  Subject  61 

2 

1 

2 

1 

2 

1 

3 

2 

2 

2 

Tables  12,  13,  and  14  show  the  responses  from  the  61  test  subjects  who  listened  to  Sound  File  “D”  at  Station  Four. 
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Chart  6.  Station  Four  Average  Question  Responses 

Chart  6  shows  the  average  scores  of  61  test  subjects  on  the  ten  item  questionnaire  at  Station  Four. 
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APPENDIX  D.  EXPERIMENT  II  DATA 


Appendix  D  provides  questionnaire  data  from  each  of  the  40  test  subjects  taken  during  Experiment  II. 


Table  15.  Average  questionnaire  scores  as  a  function  of  sound  files. 


AVERAGE  DATA  C< 

OMPARISON 

FILES/QUESTIONS 

Ql 

Q2 

Q3 

Q4 

Q5 

Q6 

Q7 

Q8 

Q9 

Q10 

SOUND  FILE  "A" 

2.85 

2.75 

2.50 

1.90 

2.50 

2.80 

3.18 

2.35 

3.43 

3.10 

SOUND  FILE  "B" 

3.53 

2.90 

2.95 

2.75 

3.53 

3.43 

3.38 

2.30 

4.38 

4.25 

SOUND  FILE  "C" 

2.83 

2.50 

2.45 

2.15 

2.15 

2.40 

3.05 

2.25 

3.48 

3.03 

SOUND  FILE  "D" 

3.05 

2.60 

2.68 

2.13 

2.45 

2.63 

2.85 

2.35 

3.68 

3.13 

Table  15  is  the  average  scores  for  each  question  for  each  sound  file.  See  Appendix  A  for  point  assignment  to  questionnaire  complete 
sentences. 
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MALE  AVERAGE 
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Chart  7.  Average  scores  are  shown  as  a  function  of  the  listener  gender. 

Chart  7  Shows  the  average  score  for  each  sound  file. 
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SOUND  FILE 

COMPARISONS 


QUESTIONS 


Chart  8.  All  Sound  File  Comparison 

Chart  8  shows  the  average  responses  for  each  question  for  each  sound  file. 
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Table  16.  Trial  1  Sound  File  "A"  Raw  Data 


SOUND  FILE 
"A" 

SURVEY  QUESTIONS  1 

SFaQx 

sfaq2 

sfaq3 

sfaq4 

sfaq5 

sfaq6 

sfaq7 

SFaQ8 

SFaQ9 

SFaQ10 

Trial  1-1 

2 

3 

3 

1 

2 

3 

4 

2 

3 

3 

Trial  1-2 

2 

3 

3 

3 

2 

3 

3 

2 

3 

3 

Trial  1-3 

3 

3 

3 

3 

2 

3 

4 

2 

4 

3 

Trial  1-4 

3 

3 

3 

3 

4 

3 

4 

3 

4 

4 

Trial  1-5 

3 

3 

3 

3 

2 

3 

4 

2 

4 

4 

Trial  1-6 

1 

1 

1 

1 

1 

1 

2 

2 

2 

1 

*Trial  1-7 

2 

3 

1 

1 

2 

3 

4 

3 

3 

3 

Trial  1-8 

3 

3 

1 

2 

3 

3 

2 

2 

4 

3 

Trial  1-9 

2 

1 

3 

2 

2 

2 

4 

2 

3 

2 

*Trial  1-10 

3 

3 

3 

2 

3 

3 

2 

2 

3 

4 

Note:  *  Indicates  Female  Test  Subject  Response 

Sound  file  “A”:  British  female  voice  (afls03b2cl7)  with  the  words:  “He  took  out  his  pipe  and  lit  up.  It  was  a  separate  bar.  ” 
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Table  17.  Trial  1  Sound  File  "B"  Raw  Data 


SOUND  FILE 
"B" 

SURVEY  QUESTIONS 

SFbQi 

sfbq2 

sfbq3 

sfbq4 

SFbQs 

sfbq6 

SFbQ7 

sfbq8 

sfbq9 

SFbQio 

Trial  1-1 

4 

3 

3 

3 

4 

4 

4 

2 

5 

4 

Trial  1-2 

3 

3 

3 

3 

4 

4 

4 

2 

4 

5 

Trial  1-3 

4 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Trial  1-4 

4 

3 

3 

3 

4 

4 

4 

3 

5 

4 

Trial  1-5 

4 

3 

3 

3 

3 

4 

4 

2 

5 

4 

Trial  1-6 

3 

1 

3 

2 

2 

2 

1 

2 

3 

3 

*Trial  1-7 

3 

3 

3 

3 

4 

4 

4 

3 

5 

5 

Trial  1-8 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

Trial  1-9 

4 

3 

3 

3 

3 

2 

4 

2 

4 

3 

*Trial  1-10 

3 

3 

3 

2 

3 

3 

2 

2 

4 

4 

Note:  *  Indicates  Female  Test  Subject  Response 

Sound  file  “B”:  British  male  voice  (amls03b2c4)  with  the  words:  “He  carried  a  bag  of  tennis  balls.  The  scheme  was  plotted  out.  ” 
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Table  18.  Trial  1  Sound  File  "C"  Raw  Data 


SOUND  FILE 
"C" 

SURVEY  QUESTIONS 

SFcQi 

SFcQ2 

SFcQ3 

SFcQ4 

SFcQs 

SFcQ6 

SFcQ7 

SFcQ8 

SFcQ9 

SFcQio 

Trial  1-1 

3 

3 

0 

2 

2 

3 

4 

2 

3 

3 

Trial  1-2 

2 

3 

3 

1 

2 

3 

3 

2 

3 

3 

Trial  1-3 

3 

3 

3 

3 

3 

3 

0 

2 

4 

4 

Trial  1-4 

4 

3 

3 

3 

2 

3 

4 

3 

4 

4 

Trial  1-5 

4 

3 

3 

3 

2 

2 

4 

2 

5 

3 

Trial  1-6 

2 

1 

1 

1 

2 

1 

2 

2 

2 

2 

*Trial  1-7 

3 

3 

3 

3 

3 

3 

3 

3 

4 

4 

Trial  1-8 

1 

1 

1 

2 

1 

1 

1 

2 

1 

1 

Trial  1-9 

2 

1 

1 

2 

1 

1 

4 

2 

2 

2 

*Trial  1-10 

3 

3 

3 

2 

3 

3 

2 

2 

4 

3 

Note:  *  Indicates  Female  Test  Subject  Response 

Sound  file  “C”:  British  female  voice  (afls01b2cl6)  with  the  words:  “You  were  the  perfect  Hostess.  Are  you  going  to  be  nice  to  me?  ” 


68 


Table  19.  Trial  1  Sound  File  "D"  Raw  Data 


SOUND  FILE 
"D" 

SURVEY  QUESTIONS 

SFdQx 

SFdQ2 

SFdQ3 

SFdQ4 

SFdQ5 

sfdq6 

SFdQ7 

SFdQ8 

SFdQ9 

SFdQio 

Trial  1-1 

3 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Trial  1-2 

3 

3 

3 

1 

3 

3 

2 

2 

3 

3 

Trial  1-3 

2 

3 

3 

0 

2 

2 

4 

2 

3 

3 

Trial  1-4 

4 

3 

3 

3 

3 

3 

4 

3 

5 

4 

Trial  1-5 

3 

3 

3 

3 

2 

2 

4 

2 

5 

3 

Trial  1-6 

2 

1 

1 

1 

1 

0 

3 

2 

1 

1 

*Trial  1-7 

3 

3 

3 

3 

3 

2 

0 

3 

3 

3 

Trial  1-8 

3 

3 

2 

1 

3 

3 

0 

3 

4 

4 

Trial  1-9 

3 

3 

3 

3 

2 

2 

4 

2 

3 

2 

*Trial  1-10 

3 

3 

3 

1 

2 

3 

2 

2 

3 

3 

Note:  *  Indicates  Female  Test  Subject  Response 

Sound  file  “D”:  British  male  voice  (amls02b2c4)  with  the  words:  “There  wasn ’t  a  scrap  of  evidence.  The  jar  was  full  of  water.  ” 

Tables  16,  17,  18,  and  19  show  the  results  from  Trial  1  and  the  responses  from  the  40  test  subjects  who  listened  to  all  sound  files  in  the 
order  of:  “A,  B,  C,  D”. 
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Chart  9.  Trial  1  Sound  File  Comparison 

Chart  9  shows  the  responses  of  the  test  subjects  by  gender. 
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Table  20.  Trial  2  Sound  File  "B"  Raw  Data 


SOUND  FILE 
"B" 

SURVEY  QUESTIONS 

SFbQi 

SFbQ2 

SFbQ3 

SFbQ4 

SFbQs 

SFbQ6 

SFbQ7 

SFbQs 

SFbQ9 

SFbQio 

Trial  2-1 

3 

3 

3 

3 

4 

4 

4 

2 

4 

5 

Trial  2-2 

2 

3 

3 

1 

3 

3 

3 

2 

3 

4 

*Trial  2-3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

Trial  2-4 

3 

3 

3 

3 

2 

3 

4 

2 

4 

4 

Trial  2-5 

3 

3 

3 

3 

3 

3 

4 

3 

4 

4 

Trial  2-6 

3 

3 

3 

2 

4 

3 

3 

2 

4 

4 

*Trial  2-7 

4 

3 

3 

3 

4 

3 

4 

2 

5 

4 

Trial  2-8 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Trial  2-9 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Trial  2-10 

3 

3 

3 

3 

3 

3 

3 

2 

4 

3 

Note:  *  Indicates  Female  Test  Subject  Response 

Sound  file  “B”:  British  male  voice  (amls03b2c4)  with  the  words:  “He  carried  a  bag  of  tennis  balls.  The  scheme  was  plotted  out.  ” 
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Table  21.  Trial  2  Sound  File  “C”  Raw  Data 


SOUND  FILE 
"C" 

SURVEY  QUESTIONS 

SFcQi 

SFcQ2 

SFcQ3 

SFcQ4 

SFcQs 

SFcQ6 

SFcQt 

SFcQ8 

SFcQ9 

SFcQio 

Trial  2-1 

3 

3 

3 

1 

3 

3 

4 

2 

3 

3 

Trial  2-2 

2 

1 

1 

1 

2 

2 

2 

2 

3 

3 

*Trial  2-3 

4 

3 

3 

3 

3 

2 

4 

3 

4 

4 

Trial  2-4 

3 

3 

3 

3 

2 

3 

4 

2 

4 

4 

Trial  2-5 

2 

3 

3 

1 

2 

2 

4 

3 

3 

3 

Trial  2-6 

3 

3 

3 

3 

1 

2 

4 

2 

4 

2 

*Trial  2-7 

4 

3 

3 

3 

2 

2 

4 

2 

5 

4 

Trial  2-8 

4 

3 

3 

3 

2 

2 

4 

2 

4 

3 

Trial  2-9 

4 

3 

3 

3 

2 

3 

4 

2 

4 

4 

Trial  2-10 

2 

3 

1 

1 

3 

3 

2 

2 

2 

3 

Note:  *  Indicates  Female  Test  Subject  Response 

Sound  file  “C”:  British  female  voice  (afls01b2cl6)  with  the  words:  “You  were  the  perfect  Hostess.  Are  you  going  to  be  nice  to  me?  ” 
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Table  22.  Trial  2  Sound  File  “D”  Raw  Data 


SOUND  FILE 
"D" 

SURVEY  QUESTIONS 

SFdOx 

SFdQ2 

SFdQ3 

SFdQ4 

SFd05 

sfdq6 

SFdQ? 

SFdQ8 

SFdQ9 

SFdQio 

Trial  2-1 

3 

3 

3 

1 

2 

3 

4 

2 

3 

3 

Trial  2-2 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

*Trial  2-3 

4 

3 

3 

3 

4 

3 

4 

4 

5 

5 

Trial  2-4 

4 

3 

3 

3 

2 

3 

4 

2 

5 

4 

Trial  2-5 

4 

3 

3 

3 

4 

4 

4 

3 

4 

4 

Trial  2-6 

3 

3 

3 

2 

1 

2 

3 

2 

3 

2 

*Trial  2-7 

4 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Trial  2-8 

4 

3 

3 

3 

4 

4 

4 

2 

5 

4 

Trial  2-9 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Trial  2-10 

2 

1 

1 

2 

2 

2 

0 

2 

2 

2 

Note:  *  Indicates  Female  Test  Subject  Response 

Sound  file  “D”:  British  male  voice  (amls02b2c4)  with  the  words:  “There  wasn ’t  a  scrap  of  evidence.  The  jar  was  full  of  water.  ” 
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Table  23.  Trial  2  Sound  File  “A”  Raw  Data 


SOUND  FILE 
"A" 

SURVEY  QUESTIONS 

SFaQi 

SFaQ2 

SFaQs 

SFaQ4 

SFaQs 

SFaQ6 

SFaQ7 

SFaQs 

SFaQ9 

SFaQio 

Trial  2-1 

3 

3 

3 

1 

2 

3 

4 

2 

3 

2 

Trial  2-2 

3 

3 

3 

3 

4 

4 

3 

2 

4 

4 

*Trial  2-3 

4 

3 

3 

3 

4 

4 

4 

4 

5 

5 

Trial  2-4 

4 

3 

3 

3 

4 

4 

4 

2 

4 

4 

Trial  2-5 

3 

3 

3 

1 

2 

3 

4 

3 

3 

3 

Trial  2-6 

4 

3 

1 

1 

1 

1 

2 

2 

2 

1 

*Trial  2-7 

3 

3 

1 

1 

2 

2 

4 

2 

3 

2 

Trial  2-8 

4 

3 

3 

3 

3 

3 

4 

2 

5 

4 

Trial  2-9 

3 

3 

3 

3 

4 

4 

2 

2 

4 

5 

Trial  2-10 

2 

3 

1 

1 

3 

3 

3 

2 

2 

3 

Note:  *  Indicates  Female  Test  Subject  Response 

Sound  file  “A”:  British  female  voice  (afls03b2cl7)  with  the  words:  “He  took  out  his  pipe  and  lit  up.  It  was  a  separate  bar.  ” 

Tables  20,  21,  22  and  23  show  the  results  of  Trial  2  and  the  responses  from  the  40  test  subjects  who  listened  to  all  sound  files  in  the 
order  of  “B,  C,  D,  A”. 
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Chart  10.  Trial  2  Sound  File  Comparison 

Chart  10  shows  the  responses  of  the  test  subjects  by  gender: 
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Table  24.  Trial  3  Sound  File  “C”  Raw  Data 


SOUND  FILE 
"C" 

SURVEY  QUESTIONS 

SFcQi 

SFcQ2 

SFcQ3 

SFcQ4 

SFcQs 

SFcQ6 

SFcQt 

SFcQs 

SFcQ9 

in 

n 

O 

o 

Trial  3-1 

3 

3 

3 

3 

3 

3 

4 

2 

4 

4 

Trial  3-2 

2 

1 

1 

2 

1 

3 

0 

2 

3 

3 

Trial  3-3 

3 

3 

3 

3 

3 

2 

4 

2 

4 

4 

Trial  3-4 

3 

3 

3 

3 

1 

2 

4 

2 

4 

2 

*Trial  3-5 

2 

1 

3 

1 

2 

2 

4 

3 

3 

2 

*Trial  3-6 

3 

3 

3 

1 

3 

3 

4 

4 

4 

3 

Trial  3-7 

3 

3 

3 

1 

2 

2 

4 

2 

4 

3 

*Trial  3-8 

2 

3 

2 

2 

2 

3 

2 

2 

3 

4 

*Trial  3-9 

3 

1 

3 

3 

2 

3 

0 

2 

4 

3 

Trial  3-10 

3 

3 

3 

3 

2 

2 

3 

3 

4 

3 

Note:  *  Indicates  Female  Test  Subject  Responses 

Sound  file  “C”:  British  female  voice  (afls01b2cl6)  with  the  words:  “You  were  the  perfect  Hostess.  Are  you  going  to  be  nice  to  me?  ” 


76 


Table  25.  Trial  3  Sound  File  “D”  Raw  Data 


SOUND  FILE 
"D" 

SURVEY  QUESTIONS 

SFDQi 

sfdq2 

sfdq3 

sfdq4 

SFdQ5 

sfdq6 

SFdQ7 

SFdQ8 

sfdq9 

SFdQio 

Trial  3-1 

3 

3 

3 

1 

2 

2 

4 

2 

3 

2 

Trial  3-2 

2 

1 

3 

1 

2 

3 

1 

2 

3 

2 

Trial  3-3 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Trial  3-4 

2 

3 

3 

1 

1 

2 

1 

2 

3 

2 

*Trial  3-5 

3 

3 

3 

3 

4 

3 

4 

4 

4 

4 

*Trial  3-6 

2 

3 

3 

1 

3 

3 

2 

4 

3 

3 

Trial  3-7 

3 

3 

3 

3 

3 

2 

4 

2 

4 

3 

*Trial  3-8 

2 

1 

2 

1 

2 

2 

4 

3 

2 

3 

*Trial  3-9 

3 

1 

3 

1 

2 

3 

0 

2 

4 

3 

Trial  3-10 

3 

3 

3 

3 

4 

3 

4 

3 

4 

4 

Note:  *  Indicates  Female  Test  Subject  Responses 

Sound  file  “D”:  British  male  voice  (amls02b2c4)  with  the  words:  “There  wasn ’t  a  scrap  of  evidence.  The  jar  was  full  of  water.  ” 
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Table  26.  Trial  3  Sound  File  “A”  Raw  Data 


SOUND  FILE 
"A" 

SURVEY  QUESTIONS 

SFaQj 

sfaq2 

sfaq3 

sfaq4 

sfaq5 

sfaq6 

sfaq7 

SFaQ8 

SFaQ9 

in 

> 

O 

o 

Trial  3-1 

3 

3 

3 

1 

3 

3 

4 

2 

3 

3 

Trial  3-2 

2 

1 

1 

1 

2 

3 

1 

2 

3 

3 

Trial  3-3 

3 

3 

3 

1 

3 

3 

4 

2 

4 

4 

Trial  3-4 

2 

3 

1 

1 

2 

2 

1 

2 

2 

2 

*Trial  3-5 

3 

3 

3 

3 

4 

3 

4 

4 

4 

4 

*Trial  3-6 

3 

3 

3 

3 

4 

4 

1 

3 

3 

4 

Trial  3-7 

3 

3 

3 

1 

2 

2 

4 

2 

4 

3 

*Trial  3-8 

3 

3 

3 

1 

2 

3 

2 

3 

4 

4 

*Trial  3-9 

3 

1 

3 

0 

1 

2 

3 

2 

3 

2 

Trial  3-10 

2 

3 

3 

3 

3 

3 

3 

3 

3 

4 

Note:  *  Indicates  Female  Test  Subject  Responses 

Sound  file  “A”:  British  female  voice  (afls03b2cl7)  with  the  words:  “He  took  out  his  pipe  and  lit  up.  It  was  a  separate  bar.  ” 
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Table  27.  Trial  3  Sound  File  “B”  Raw  Data 


SOUND  FILE 
"B" 

SURVEY  QUESTIONS 

SFbQi 

sfbq2 

sfbq3 

sfbq4 

SFbQs 

sfbq6 

SFbQ7 

SFbQs 

sfbq9 

SFbQio 

Trial  3-1 

3 

3 

3 

3 

4 

3 

4 

2 

5 

4 

Trial  3-2 

3 

1 

1 

1 

2 

3 

1 

2 

3 

3 

Trial  3-3 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Trial  3-4 

3 

3 

3 

3 

3 

3 

4 

2 

4 

3 

*Trial  3-5 

4 

3 

3 

3 

4 

3 

4 

4 

4 

4 

*Trial  3-6 

4 

3 

3 

3 

4 

4 

2 

3 

5 

5 

Trial  3-7 

4 

3 

3 

3 

4 

3 

4 

2 

5 

4 

*Trial  3-8 

3 

3 

3 

2 

2 

3 

3 

3 

4 

4 

*Trial  3-9 

4 

3 

3 

3 

4 

4 

4 

2 

5 

4 

Trial  3-10 

3 

3 

3 

3 

4 

3 

3 

3 

4 

4 

Note:  *  Indicates  Female  Test  Subject  Responses 

Sound  file  “B”:  British  male  voice  (amls03b2c4)  with  the  words:  “He  carried  a  bag  of  tennis  balls.  The  scheme  was  plotted  out.  ” 

Tables  24,  25,  26  and  27  show  the  results  of  Trial  3  and  the  responses  from  the  40  test  subjects  who  listened  to  all  sound  files  in  the 
order  of  “C,  D,  A,  B 
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r 


TRIAL  3 


A 


AVERAGE  COMPARISON 


FILET"  FILET"  FILE  "A"  FILET" 
SOUND  FILES 


Chart  11.  Trial  3  Sound  File  Comparison 

Chart  1 1  shows  the  responses  of  the  test  subjects  by  gender. 
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Table  28.  Trial  4  Sound  File  “D”  Raw  Data 


SOUND  FILE 
"D" 

SURVEY  QUESTIONS 

SFdQ! 

SFdQ2 

SFdQ3 

SFdQ4 

SFdQ5 

SFdQ6 

SFdQv 

SFdQs 

SFdQ9 

SFdQio 

*Trial  4-1 

3 

3 

1 

1 

2 

2 

4 

2 

3 

2 

*Trial  4-2 

2 

1 

0 

1 

2 

3 

2 

0 

3 

3 

Trial  4-3 

3 

1 

2 

3 

2 

3 

0 

2 

3 

2 

Trial  4-4 

3 

3 

3 

1 

2 

3 

2 

2 

4 

3 

*Trial  4-5 

2 

3 

3 

3 

1 

1 

4 

3 

3 

2 

Trial  4-6 

3 

1 

2 

3 

1 

3 

0 

2 

3 

2 

Trial  4-7 

4 

3 

3 

1 

2 

2 

4 

3 

5 

3 

*Trial  4-8 

4 

3 

3 

3 

1 

1 

0 

3 

5 

2 

*Trial  4-9 

3 

3 

3 

3 

2 

2 

4 

2 

4 

4 

Trial  4-10 

3 

3 

3 

3 

2 

3 

4 

3 

4 

3 

Note:  *  Indicates  Female  Test  Subject  Responses 

Sound  file  “D”:  British  male  voice  (amls02b2c4)  with  the  words:  “There  wasn ’t  a  scrap  of  evidence.  The  jar  was  full  of  water.  ” 


81 


Table  29.  Trial  4  Sound  File  “A”  Raw  Data 


SOUND  FILE 
"A" 

SURVEY  QUESTIONS 

SFaQi 

sfaq2 

sfaq3 

sfaq4 

SFaQ5 

sfaq6 

sfaq7 

sfaq8 

sfaq9 

sfaq10 

*Trial  4-1 

3 

3 

1 

1 

2 

2 

3 

2 

3 

2 

*Trial  4-2 

3 

3 

3 

0 

4 

4 

3 

0 

4 

4 

Trial  4-3 

3 

1 

3 

1 

2 

3 

3 

2 

2 

2 

Trial  4-4 

3 

3 

1 

4 

3 

2 

2 

4 

4 

*Trial  4-5 

2 

3 

3 

3 

2 

2 

3 

3 

3 

3 

Trial  4-6 

2 

3 

3 

1 

2 

3 

3 

2 

3 

2 

Trial  4-7 

4 

3 

3 

1 

2 

3 

4 

3 

4 

3 

*Trial  4-8 

3 

3 

3 

3 

1 

1 

4 

3 

5 

2 

*Trial  4-9 

4 

3 

3 

3 

2 

3 

4 

2 

4 

4 

Trial  4-10 

3 

3 

3 

3 

2 

3 

4 

3 

4 

3 

Note:  *  Indicates  Female  Test  Subject  Responses 

Sound  file  “A”:  British  female  voice  (afls03b2cl7)  with  the  words:  “He  took  out  his  pipe  and  lit  up.  It  was  a  separate  bar.  ” 
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Table  30.  Trial  4  Sound  File  “B”  Raw  Data 


SOUND  FILE 
"B" 

SURVEY  QUESTIONS 

SFbQi 

SFbQ2 

SFbQ3 

SFbQ4 

SFbQ5 

SFbQ6 

SFbQ7 

SFbQ8 

SFbQ9 

SFbQio 

*Trial  4-1 

4 

3 

3 

3 

4 

3 

4 

2 

4 

5 

*Trial  4-2 

3 

3 

3 

1 

4 

3 

1 

0 

4 

4 

Trial  4-3 

3 

3 

3 

3 

2 

4 

3 

2 

5 

5 

Trial  4-4 

4 

3 

3 

3 

4 

4 

2 

2 

4 

5 

*Trial  4-5 

4 

3 

3 

3 

4 

3 

4 

3 

4 

4 

Trial  4-6 

4 

3 

3 

3 

4 

4 

1 

2 

5 

5 

Trial  4-7 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

*Trial  4-8 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

*Trial  4-9 

4 

3 

3 

3 

4 

4 

4 

2 

5 

5 

Trial  4-10 

4 

3 

3 

3 

4 

4 

4 

3 

5 

5 

Note:  *  Indicates  Female  Test  Subject  Responses 

Sound  file  “B”:  British  male  voice  (amls03b2c4)  with  the  words:  “He  carried  a  bag  of  tennis  balls.  The  scheme  was  plotted  out.  ” 
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Table  31.  Trial  4  Sound  File  “C”  Raw  Data 


SOUND  FILE  "C" 

SURVEY  QUESTIONS 

SFcQi 

SFcQ2 

SFcQ3 

SFcQ4 

SFcQ5 

SFcQ6 

SFcQ7 

SFcQs 

SFcQ9 

SFcQio 

*Trial  4-1 

2 

3 

1 

2 

2 

2 

3 

2 

3 

3 

*Trial  4-2 

2 

1 

1 

1 

2 

2 

2 

0 

2 

2 

Trial  4-3 

3 

3 

3 

2 

2 

2 

3 

2 

3 

3 

Trial  4-4 

2 

1 

1 

1 

2 

3 

2 

2 

3 

3 

*Trial  4-5 

3 

3 

3 

3 

2 

2 

4 

3 

4 

3 

Trial  4-6 

3 

1 

3 

1 

2 

2 

2 

2 

3 

2 

Trial  4-7 

4 

3 

3 

3 

3 

3 

4 

3 

4 

3 

*Trial  4-8 

3 

3 

3 

3 

2 

3 

3 

3 

4 

3 

*Trial  4-9 

3 

3 

3 

3 

3 

2 

4 

2 

4 

3 

Trial  4-10 

3 

3 

3 

1 

2 

3 

3 

3 

4 

3 

Note:  *  Indicates  Female  Test  Subject  Responses 

Sound  file  “C”:  British  female  voice  (afls01b2cl6)  with  the  words:  “You  were  the  perfect  Hostess.  Are  you  going  to  be  nice  to  me?  ” 

Tables  28,  29,  30  and  31  show  the  results  of  Trial  3  and  the  responses  from  the  40  test  subjects  who  listened  to  all  sound  files  in  the 
order  of“D,  A,  B,  C. 
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TRIAL  4 

AVERAGE  COMPARISON 


OVERALL 

AVERAGE 

FEMALE 

AVERAGE 

MALE  AVERAGE 


FILE"D"  FILE  "A"  FILE  r,B,f  FILE"C" 


SOUND  FILES 


4 


Chart  12.  Trial  4  Sound  File  Comparison 


Chart  12  shows  the  responses  of  the  test  subjects  by  gender 
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APPENDIX  E  MULTI-ATTRIBUTE  ANALYSIS 


Appendix  E  shows  the  comparison  of  two  similar  questions  which  illustrates  how  the  test  subjects  perceived  quality  of  each  of  the 
sound  fdes  at  the  various  stations. 


Table  32.  Question  3  &  4  Raw  Data 


STATION  ONE 

STATION  TWO 

STATION  THREE 

STATION  FOUR 

Test 

Subject 

SiQ3 

S1Q4 

s2q3 

s2cu 

5  3^3 

s3c^ 

54Q4 

TS-1 

1 

2 

3 

2 

1 

2 

3 

3 

TS-Z 

3 

1 

3 

3 

O 

1 

3 

1 

TS-3 

2 

1 

3 

3 

3 

2 

3 

3 

TS-4 

3 

3 

3 

3 

3 

3 

3 

3 

TS-5 

3 

3 

3 

3 

3 

3 

3 

3 

TS-6 

1 

1 

3 

3 

3 

1 

3 

1 

TS-7 

3 

1 

3 

2 

3 

Z 

1 

1 

TS-S 

1 

1 

3 

3 

3 

O 

1 

1 

TS-9 

3 

3 

3 

3 

3 

3 

3 

3 

TS-IO 

3 

1 

3 

1 

3 

1 

3 

2 

TS-1 1 

3 

3 

3 

O 

3 

1 

3 

1 

TS-1  Z 

3 

3 

3 

3 

2 

1 

3 

3 

TS-13 

1 

1 

3 

3 

3 

2 

3 

3 

TS-14 

3 

3 

3 

3 

3 

3 

3 

3 

TS-15 

2 

1 

3 

3 

O 

1 

3 

O 

TS-1G 

3 

3 

3 

3 

3 

1 

3 

1 

TS-17 

2 

1 

3 

3 

3 

1 

3 

3 

TS-1S 

1 

1 

3 

3 

3 

3 

3 

3 

TS-19 

3 

3 

3 

3 

3 

3 

2 

1 

TS-ZO 

2 

1 

3 

3 

3 

2 

3 

3 
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Table  33.  Question  3  &  4  Raw  Data  (cont) 


STATION  ONE 

STATION  TWO 

STATIO 

VJ  THREE 

STATIO 

N  FOUR 

Test 

Subject 

SiQ3 

S1Q4 

S2Q3 

S2Q4 

S3Q3 

S3Q4 

s*Q3 

S4Q4 

TS-21 

1 

2 

3 

2 

1 

3 

3 

3 

TS-22 

1 

1 

3 

3 

* 

1 

1 

1 

TS-23 

1 

3 

3 

3 

2 

1 

3 

2 

TS-24 

1 

1 

3 

3 

3 

3 

3 

3 

TS-25 

3 

3 

3 

3 

3 

3 

3 

3 

TS-26 

3 

3 

3 

3 

3 

1 

3 

3 

TS-27 

1 

3 

3 

3 

3 

2 

3 

3 

TS-2S 

2 

1 

3 

3 

3 

3 

3 

3 

TS-29 

3 

1 

3 

3 

3 

1 

3 

1 

TS-30 

3 

3 

3 

3 

3 

3 

3 

3 

TS-31 

3 

1 

3 

3 

3 

3 

3 

3 

TS-32 

3 

1 

3 

3 

3 

3 

3 

1 

TS-33 

2 

1 

3 

3 

3 

1 

3 

2 

TS-34 

0 

1 

3 

3 

3 

2 

1 

1 

TS-35 

3 

3 

3 

3 

3 

3 

3 

3 

TS-36 

3 

3 

3 

3 

3 

3 

3 

3 

TS-37 

3 

1 

3 

1 

3 

2 

3 

1 

TS-38 

3 

3 

3 

3 

3 

3 

3 

2 

TS-39 

3 

1 

3 

3 

0 

1 

0 

1 

TS-40 

1 

2 

3 

3 

1 

1 

1 

1 
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Table  34.  Question  3  &  4  Raw  Data  (cont) 


STATION  ONE 

STATION  TWO 

STATION  THREE 

STATION  FOUR 

Test 

Subject 

s,Q3 

S.Q4 

s2Q3 

s2Q4 

^3^3 

S304 

s4Q3 

^4^4 

TS-41 

3 

3 

3 

3 

3 

3 

1 

1 

TS-42 

3 

3 

3 

3 

1 

1 

3 

3 

TS-43 

3 

1 

1 

1 

3 

2 

3 

2 

TS-44 

1 

1 

3 

3 

3 

2 

3 

1 

TS-45 

3 

3 

3 

3 

1 

1 

3 

2 

TS-46 

3 

1 

3 

3 

1 

2 

3 

3 

TS-47 

3 

1 

3 

3 

3 

3 

3 

3 

TS-4S 

3 

3 

3 

3 

3 

3 

3 

3 

TS-49 

3 

3 

3 

3 

3 

3 

3 

3 

TS-SO 

1 

1 

3 

3 

1 

2 

2 

3 

TS-51 

3 

3 

3 

3 

3 

2 

3 

1 

TS-5  2 

1 

1 

3 

3 

1 

1 

1 

2 

TS-53 

3 

3 

3 

3 

3 

3 

3 

3 

TS-54 

3 

3 

3 

3 

3 

1 

3 

3 

TS -55 

1 

1 

3 

2 

3 

2 

3 

2 

TS-56 

3 

3 

3 

3 

3 

3 

3 

3 

TS-57 

3 

3 

3 

3 

3 

3 

1 

1 

TS-5S 

2 

1 

3 

3 

3 

2 

3 

1 

TS-59 

3 

3 

3 

3 

3 

1 

3 

3 

TS-50 

1 

1 

3 

3 

1 

2 

3 

2 

TS-5 1 

2 

1 

3 

3 

1 

1 

2 

1 

Tables  32,  33  and  34  show  the  responses  from  the  61  test  subjects  at  each  Station  for  questions  3  and  4. 
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Figure  10.  Questions  3  &  4  Station  One 
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Figure  11.  Questions  3  &  4  Station  Two 
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Figure  12.  Questions  3  &  4  Station  Three 
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Figure  13.  Questions  3  &  4  Station  Four 

Figures  10,  11,  12  and  13  show  the  percentage  of  responses  from  the  61  test  subjects  within  each  quadrant. 


93 


Table  35.  Questions  9  &  1  Raw  Data 


STATION  ONE 

STATION  TWO 

STATION  THREE 

STATION  FOUR 

Test 

Subject 

siQi 

S,Qn 

S,Qn 

S.O, 

TS-1 

3 

2 

4 

3 

2 

2 

5 

3 

TS-Z 

3 

7 

4 

4 

9 

9 

4 

3 

1S-3 

'7 

1 

4 

4 

9 

4 

-9p 

TS-4 

5 

4 

5 

4 

5 

3 

3 

4 

TS-5 

5 

4 

5 

4 

5 

4 

5 

4 

TS-6 

3 

1 

5 

4 

5 

9 

s 

3 

TS-7 

3 

7 

7 

2 

2 

2 

i 

1 

TS-8 

■-> 

j 

JT 

4 

4 

4 

4 

3 

a 

JT 

TS-9 

5 

4 

5 

4 

4 

-np 

4 

TS-IO 

3 

7 

4 

■-> 

4 

■Ti 

3 

4 

•-> 

3 

TS-1 1 

4 

3 

4 

3 

■-> 

3 

3 

9 

TS-1 2 

4 

4 

5 

4 

7 

5 

4 

TS-1  3 

3 

1 

4 

4 

9 

5 

4 

TS  14 

4 

3 

4 

4 

5 

4 

5 

4 

TS-1 5 

9 

i 

4 

3 

3 

9 

3 

3 

TS-1 6 

4 

3 

4 

4 

7 

9 

9 

9 

TS-1 7 

3 

■7 

4 

4 

4 

3 

4 

-* 

3 

TS-1 8 

9 

9 

5 

4 

5 

4 

4 

3 

Ts-i  n 

5 

4 

5 

4 

4 

Ti 

3 

9 

i 

TS  ZO 

3 

9 

5 

4 

3 

3 

4 

4 
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Table  36.  Questions  9  &  1  Raw  Data  (cont) 


STATION  ONE 

STATION  TWO 

STATION  THREE 

STATION  FOUR 

Test 

Subject 

SnQ0 

siQi 

SiQi 

S,Qn 

S,Q0 

S,Qn 

TS-21 

3 

■7 

4 

3 

4 

3 

4 

3 

TS-2  2 

4 

7 

5 

4 

-> 

3 

7 

3 

7 

TS-23 

4 

3 

5 

4 

3 

3 

4 

3 

TS-24 

4 

3 

5 

4 

5 

4 

4 

a 

3 

TS-25 

4 

3 

5 

3 

5 

3 

5 

4 

TS-26 

4 

■-> 

3 

4 

4 

•-> 

3 

7 

4 

3 

TS-27 

3 

7 

5 

4 

4 

■-> 

3 

5 

4 

TS-2S 

7 

7 

4 

T( 

3 

4 

3 

4 

3 

TS-29 

4 

4 

4 

"■> 

3 

3 

3 

4 

--*> 

3 

TS-30 

4 

4 

4 

4 

5 

4 

5 

4 

TS-31 

3 

3 

3 

3 

4 

4 

4 

3 

TS-3  2 

3 

7 

5 

4 

4 

■Ti 

3 

4 

■nil 

3 

TS-33 

4 

3 

5 

4 

4 

3 

3 

7 

TS-34 

7 

7 

4 

3 

4 

3 

3 

3 

TS-35 

5 

4 

5 

4 

5 

4 

5 

4 

TS-36 

4 

3 

5 

4 

3 

3 

4 

-> 

3 

TS-37 

3 

7 

3 

3 

3 

3 

9 

1 

TS-3S 

5 

4 

5 

4 

5 

4 

5 

4 

TS-39 

3 

3 

4 

3 

3 

7 

-% 

3 

7 

TS-40 

4 

3 

5 

4 

3 

3 

7 

7 
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Table  37.  Questions  9  &  1  Raw  Data  (cont) 


STATIO  N  ON  E 

STATION  TWO 

STATION  THREE 

STATION  FOUR 

Test 

Subject 

SnQn 

S,Q„ 

S-nQt 

S^Qn 

S4Q1 

TS-41 

4 

3 

S 

4 

4 

4 

7 

TS-42 

5 

4 

4 

3 

3 

2 

4 

3 

TS-43 

3 

3 

2 

1 

3 

3 

1 

1 

TS-44 

3 

3 

4 

3 

3 

2 

2 

2 

TS-45 

4 

3 

4 

3 

3 

2 

3 

3 

TS-46 

3 

2 

4 

4 

4 

2 

5 

4 

TS-47 

4 

3 

4 

4 

4 

3 

4 

3 

TS-48 

5 

1 

5 

4 

5 

4 

4 

3 

TS-49 

4 

3 

4 

3 

4 

3 

4 

3 

TS-SO 

3 

2 

3 

3 

2 

2 

3 

3 

TS-51 

5 

4 

5 

4 

3 

2 

3 

3 

TS-52 

3 

2 

4 

3 

3 

2 

3 

TS-53 

3 

3 

3 

3 

4 

3 

4 

3 

TS-54 

5 

4 

5 

4 

4 

3 

4 

3 

TS-55 

3 

2 

4 

3 

4 

3 

4 

3 

TS-5G 

4 

3 

4 

4 

5 

4 

4 

3 

TS-57 

4 

4 

4 

3 

4 

3 

i 

1 

TS-58 

4 

3 

5 

4 

4 

4 

4 

3 

TS-59 

4 

2 

5 

4 

4 

3 

5 

4 

TS-GO 

2, 

2 

4 

3 

3 

2 

3 

2 

TS-G1 

3 

2 

4 

3 

2 

2 

2 

2 

Tables  35,  36and  37  show  the  responses  from  the  61  test  subjects  at  each  Station  for  questions  9  and  1. 
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Figure  14.  Questions  9  &  1  Station  One 
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Figure  15.  Questions  9  &  1  Station  Two 
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Figure  16.  Questions  9  &  1  Station  Three 
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Figure  17.  Questions  9  &  1  Station  Four 

Figures  14,  15,  16  and  17  show  the  percentage  of  responses  from  the  61  test  subjects  within  each  quadrant. 


100 


Table  38.  Questions  10  &  5  Raw  Data 


STATION  ONE 

STATION  TWO 

STATION  THREE 

STATION  FOUR 

TEST 

SUBJECTS 

Qin 

s,q5 

EsQin 

S,Qs 

^/tQin 

S4Q111 

TS-1 

4 

2 

4 

3 

2 

2 

4 

4 

TS-2 

2 

2 

5 

4 

2 

2 

3 

3 

TS-3 

1 

1 

3 

3 

4 

1 

4 

3 

TS-4 

2 

2 

5 

4 

2 

2 

3 

2 

T  S-5 

1 

1 

4 

4 

4 

4 

4 

3 

TS-6 

2 

1 

5 

4 

3 

2 

3 

3 

TS-7 

3 

2 

4 

3 

3 

2 

1 

1 

TS-8 

3 

2 

4 

4 

4 

2 

3 

2 

TS-9 

4 

3 

4 

3 

4 

2 

4 

3 

TS-1  0 

3 

2 

4 

4 

3 

2 

3 

3 

TS-1 1 

4 

3 

4 

4 

3 

2 

2 

2 

TS-1  2 

4 

3 

5 

4 

3 

2 

5 

3 

TS-1  3 

1 

1 

5 

4 

3 

2 

5 

4 

TS  1-1 

'I 

2 

fl 

3 

4 

3 

3 

TS-1  5 

1 

1 

4 

2 

2 

1 

3 

3 

TS-1  6 

3 

2 

3 

2 

1 

1 

1 

I 

TS-17 

3 

2 

4 

3 

3 

2 

4 

3 

TS-1  8 

2 

1 

3 

4 

4 

3 

2 

2 

TS-1  9 

5 

2 

5 

4 

4 

2 

2 

1 

TS-20 

3 

2 

4 

2 

3 

2 

4 

2 
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Table  39.  Questions  10  &  5  Raw  Data  (cont) 


STATION  ONE 

STATION  TWO 

STATION  THREE 

STATION  FOUR 

TEST 

SUBJECTS 

siQio 

SiQs 

^2  Qio 

S2Qs 

S3Q10 

S4QlO 

S4Qs 

TS-21 

2 

2 

4 

2 

3 

2 

4 

2 

TS-22 

3 

2 

5 

4 

3 

2 

2 

2 

TS-23 

3 

2 

5 

4 

4 

3 

3 

2 

TS-24 

3 

2 

5 

4 

5 

4 

4 

3 

TS-25 

3 

2 

4 

3 

4 

3 

5 

4 

TS-26 

3 

3 

4 

4 

3 

2 

4 

3 

TS-27 

4 

4 

5 

4 

4 

4 

4 

4 

TS-28 

2 

1 

4 

4 

4 

4 

3 

3 

TS-29 

4 

3 

4 

3 

3 

2 

3 

2 

TS-30 

4 

2 

5 

4 

4 

2 

5 

4 

TS-31 

3 

2 

3 

2 

4 

4 

4 

3 

TS-32 

4 

4 

5 

4 

5 

4 

5 

4 

TS-33 

4 

4 

4 

3 

3 

3 

3 

2 

TS-34 

2 

2 

4 

3 

2 

2 

3 

2 

TS-35 

4 

3 

5 

4 

5 

4 

5 

4 

TS-36 

4 

2 

4 

4 

3 

2 

4 

3 

TS-37 

3 

2 

3 

3 

3 

2 

2 

1 

TS-38 

5 

4 

5 

4 

3 

3 

5 

4 

TS-39 

3 

2 

3 

2 

2 

1 

3 

2 

TS-40 

3 

2 

4 

4 

3 

2 

2 

2 
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Table  40.  Questions  10  &  5  Raw  Data  (cont.) 


STATION  ONE 

STATION  TWO 

STATION  THREE 

STATION  FOUR 

TEST 

SUBJECTS 

?>i  Om 

f>!  Qs 

TS-41 

4 

4 

5 

4 

2 

2 

1 

1 

TS-4  2 

4 

3 

4 

4 

2 

2 

3 

3 

TS-4  3 

3 

3 

2 

1 

3 

2 

2 

1 

TS-4  4 

3 

2 

4 

3 

2 

2 

1 

1 

TS-4i> 

4 

2 

5 

4 

4 

4 

4 

4 

TS-4  5 

2 

1 

4 

3 

3 

2 

4 

3 

To-47 

2 

1 

4 

2 

2 

1 

2 

1 

TS-4  3 

3 

1 

5 

4 

2 

1 

3 

2 

TS-4  3 

4 

3 

5 

4 

3 

4 

4 

4 

TS-53 

3 

2 

3 

2 

2 

1 

3 

3 

TS-51 

3 

2 

5 

4 

2 

1 

3 

2 

TS-52 

3 

2 

4 

4 

1 

1 

2 

2 

TS-53 

3 

2 

4 

2 

3 

2 

3 

2 

TS  5-1 

5 

4 

5 

4 

3 

2 

4 

2 

TS-55 

2 

2 

3 

2 

3 

2 

4 

2 

TS -55 

4 

3 

4 

4 

5 

4 

4 

4 

TS-57 

4 

4 

4 

3 

4 

3 

1 

1 

TS-53 

4 

3 

5 

4 

3 

2 

2 

1 

TS-53 

3 

2 

5 

4 

4 

3 

5 

4 

TS -63 

3 

2 

4 

3 

3 

2 

4 

3 

TS-61 

2 

2 

4 

3 

2 

1 

2 

2 

Tables  38,  39  and  40  show  the  responses  from  the  61  test  subjects  at  each  Station  for  questions  10  and  5. 
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Figure  18.  Questions  10  &  5  Station  One 

104 


Figure  19.  Questions  10  &  5  Station  Two 
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Figure  20.  Questions  10  &  5  Station  Three 
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Figure  21.  Questions  10  &  5  Station  Four 


Figures  18,  19,  20  and  21  show  the  percentage  of  responses  from  the  61  test  subjects  within  each  quadrant. 
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APPENDIX  F  THESIS  PROPOSAL  AND  IRB  APPROVAL 

FORMS 


Appendix  F  consists  of  the  approved  thesis  proposal  and  institutional  review  board  (IRB) 
forms. 


VIFMORaNTHTM 

1'ioni:  LCDR  COY  M.  ADAMS,  JR. 


Du;e:  May  20  >  >.007 


To:  Projiiatr:  (HJloer.  JnJ^orramlit^ri  xiul  Operation?,  CurriculniR  .>56 

V»i; 

(  I  i  Tf^sis  Aiivijciir.  DR.  CYNTHLA  1&V1NK 
( 2 1  T( vest kCi^-A  il vixor ,  DR ,  KPL SON I KV  IMis 

C  3  *  Ac&tem  i  c  Asm  ici  nie,  1  r .  iinnn«u  iy n  Systems  and  O  perat  Lo ns  Cur c  iculiitn 

(4j  t.'Lu.ii  nuui,  Eitloi rriaiii>n  SciciiLCi  Ffeji}jiimc:nl 

Suhj:  THESIS  PROPOSAL  ICO  LCDR  COV  \JONKttr  ADAMS.  JR.  24 1 -04-0988. 
USN-1  110 


Fii-jI;  f  i)  Tlio£i&  Proposal 


1 .  Teutative  TiLlc  of  Pmpusctl  'itwsis:  VgJP  equality  MeaKuremerits.  in  A  VlulEilevd 
Secure  CM I.S)  lir.vironmcni 

2,  GaicraJ  tif  lYujjowd  Uie*is  Multilevel  Secure  (MLSJ  KiivLionniem 

a-  l  expo  Cl  my  thesis  wiil  be  untlassilirtl 


0-1  H. /t>=F- 


y  Du  it  .■ 


t/vA 

Quit: 


!;  </7 

E>alc 

J"  jrvkJc  ~~ 
Date 

{A/A-*  4- 

Dstr  1 


1 


Figure  22.  Copy  of  Approved  Thesis  Proposal 
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Naval  Postgraduate  School 
Institutional  Review  Board  (IRB) 


28-Sep-07 


From:  LT  Brent  Qldc,  P h.Q. 

To:  Professor  Cynthia  Irvine 

LCDR  Coy  M,  Adams 


Subject:  YOUR  PROJECT:  VOICE  QUALITY  FOR  VOICE  OVER  INTERNET 

PROTOCOL  (VOIP)  IN  A  MULTILEVEL  SECURE  NETWORK  (MLS) 

L  The  NFS  IRB  is  pleased  to  inform  you  that  the  NPS  Institutional  Review 
Beard  has  approved  your  project  (NPS  IRBfc  NPS20G70I03JR-EP7-A). 

2.  The  NPS  IRB  was  originally  certified  by  SLIMED  on  26  July  2002  ind 
has  been  rc-cmified  ur(til  30  November  2007. 

3 .  Thi*  approval  is  valid  for  one  year  from  this  dale*  Plwae  submit  a  copy  cf 
all  records  and  consent  forms  to  the  Research  and  Sponsored  Programs 
Office  (Laura  Ann  ikner-  Price,  Halljgan  Hall,  Room  201 B)  atihe 
conclusion  of  this  project. 

4.  If  your  protocol  changes  at  any  time,  you  will  need  io  resubmit  your 
project  proposal  to  the  NFS  IRB, 


Sincerely, 


Li  Brent  Qlde,  Ph.D. 

Chair 

NPS  Institutional  Review  Board 


Figure  23.  Institutional  Review  Board  (IRB)  Approval 
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APPLICATION  FOR 
HUMAN  SUBJECTS  REVIEW  (HSR) 


NFS  IRB  NUMBER  (to  be  assigned  by  RSPO) 


PRINCIPAL  INVESTIGATOR^)  (Full  Name,  Cade,  Telephone) 

Dr.  Cynthia  Irvine,  CS  Department,  93 1-55(3-2461 

Co-uimtiEator:  LCDR  Coy  M.  Adams,  831-393-9382 _ 

TITLE  OF  EXPERIMENT 

Quality  of  voice  transmission  for  Voice  over  Internet  Protocol  (VoIP)  in  a  multilevel  secure 
{MLS}  network. _ 

APPROVAL  REQUESTED  [X]  New  [  ]  Renewal  [  ]  Amendment 

LEVEL  OF  RISK  [  ]  Exempt  [X]  Minimal  [  ]  More  than  Minimal 
All  methods  are  umn -invasive  and  mirror  typical  daily  activities  (e.g.  phone  conversation)  a 
student  might  take  part  in  tilth  those  conducting  research.  The  student  will  wear  a  hand 
free  PC  headset  with  one  ear  piece  covering  the  ear  (nothing  inserted)  with  a  microphone 
a  I'm  attached,  which  the  test  subject  wiU  not  nse.  This  test  is  for  listening  observation  oniv. 

WORK  WILL  BE  DONE  IN  (Site.Bldg.Rm)  ESTIMATED  NUMBER  OF  DAYS  TO 

COMPLETE 

_ GE-B04 _ 45 _ 

MAXIMUM  NUMBER  OF  SUBJECTS  ESTIMATED  LENGTH  OF  EACH 

SUBJECT’S  PARTICIPATION: 

_ 100 _ 30-60 Minutes  (approximately) _ 

SPECIAL  POPULATIONS  THAT  WILL  BE  USED  AS  SUBJECTS 

[  ]  Subordinates  [  ]  Minors  [X]  NFS  Students  [  ]  Special  Needs  (e.g.  Pregnant  women) 
Specify  safeguards  to  avoid  undue  influence  and  protect  subject's  rights: 

No  identifying  information  will  he  recorded  or  retained  on  the  test  subjects  for  the  purpose 
of  thesis  research;  names  are  included  on  the  questionnaire  for  bookkeeping  purposes 
ONLY.  Volunteers  will  be  provided  a  verbal  brief  on  testing  approach  and  subject  rights 
in  addition  to  a  written  consent  form. _ 

SCIENTIFIC  MERIT  REVIEW  (Check  all  that  apply) 

[  ]  This  research  is  part  of  a  funded  project  (Job  Order  Number  ) 

[X]  This  research  is  a  student  thesis  (Attach  a  copy  of  the  approved  thesis  proposal) 

[  X  ]  Other  {Attach  a  complete  research  proposal) 

OUTSIDE  COOPERATING  INVESTIGATORS  AND  AGENCIES 

NONE  APPLICABLE 


Figure  24.  Institutional  Review  Board  (IRB)  Application 
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